Alpha Software Mobile Development Tools:   Alpha Anywhere    |   Alpha TransForm subscribe to our YouTube Channel  Follow Us on LinkedIn  Follow Us on Twitter  Follow Us on Facebook

Announcement

Collapse

The Alpha Software Forum Participation Guidelines

The Alpha Software Forum is a free forum created for Alpha Software Developer Community to ask for help, exchange ideas, and share solutions. Alpha Software strives to create an environment where all members of the community can feel safe to participate. In order to ensure the Alpha Software Forum is a place where all feel welcome, forum participants are expected to behave as follows:
  • Be professional in your conduct
  • Be kind to others
  • Be constructive when giving feedback
  • Be open to new ideas and suggestions
  • Stay on topic


Be sure all comments and threads you post are respectful. Posts that contain any of the following content will be considered a violation of your agreement as a member of the Alpha Software Forum Community and will be moderated:
  • Spam.
  • Vulgar language.
  • Quotes from private conversations without permission, including pricing and other sales related discussions.
  • Personal attacks, insults, or subtle put-downs.
  • Harassment, bullying, threatening, mocking, shaming, or deriding anyone.
  • Sexist, racist, homophobic, transphobic, ableist, or otherwise discriminatory jokes and language.
  • Sexually explicit or violent material, links, or language.
  • Pirated, hacked, or copyright-infringing material.
  • Encouraging of others to engage in the above behaviors.


If a thread or post is found to contain any of the content outlined above, a moderator may choose to take one of the following actions:
  • Remove the Post or Thread - the content is removed from the forum.
  • Place the User in Moderation - all posts and new threads must be approved by a moderator before they are posted.
  • Temporarily Ban the User - user is banned from forum for a period of time.
  • Permanently Ban the User - user is permanently banned from the forum.


Moderators may also rename posts and threads if they are too generic or do not property reflect the content.

Moderators may move threads if they have been posted in the incorrect forum.

Threads/Posts questioning specific moderator decisions or actions (such as "why was a user banned?") are not allowed and will be removed.

The owners of Alpha Software Corporation (Forum Owner) reserve the right to remove, edit, move, or close any thread for any reason; or ban any forum member without notice, reason, or explanation.

Community members are encouraged to click the "Report Post" icon in the lower left of a given post if they feel the post is in violation of the rules. This will alert the Moderators to take a look.

Alpha Software Corporation may amend the guidelines from time to time and may also vary the procedures it sets out where appropriate in a particular case. Your agreement to comply with the guidelines will be deemed agreement to any changes to it.



Bonus TIPS for Successful Posting

Try a Search First
It is highly recommended that a Search be done on your topic before posting, as many questions have been answered in prior posts. As with any search engine, the shorter the search term, the more "hits" will be returned, but the more specific the search term is, the greater the relevance of those "hits". Searching for "table" might well return every message on the board while "tablesum" would greatly restrict the number of messages returned.

When you do post
First, make sure you are posting your question in the correct forum. For example, if you post an issue regarding Desktop applications on the Mobile & Browser Applications board , not only will your question not be seen by the appropriate audience, it may also be removed or relocated.

The more detail you provide about your problem or question, the more likely someone is to understand your request and be able to help. A sample database with a minimum of records (and its support files, zipped together) will make it much easier to diagnose issues with your application. Screen shots of error messages are especially helpful.

When explaining how to reproduce your problem, please be as detailed as possible. Describe every step, click-by-click and keypress-by-keypress. Otherwise when others try to duplicate your problem, they may do something slightly different and end up with different results.

A note about attachments
You may only attach one file to each message. Attachment file size is limited to 2MB. If you need to include several files, you may do so by zipping them into a single archive.

If you forgot to attach your files to your post, please do NOT create a new thread. Instead, reply to your original message and attach the file there.

When attaching screen shots, it is best to attach an image file (.BMP, .JPG, .GIF, .PNG, etc.) or a zip file of several images, as opposed to a Word document containing the screen shots. Because Word documents are prone to viruses, many message board users will not open your Word file, therefore limiting their ability to help you.

Similarly, if you are uploading a zipped archive, you should simply create a .ZIP file and not a self-extracting .EXE as many users will not run your EXE file.
See more
See less

How can I make this function run quicker

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    How can I make this function run quicker

    The following function checks the passed field [VarLine] to see if it has the chars/digits 0-9,A-F, ie its a 6 byte HexValue. It works but takes hours to run against 47000+ records.

    Is there a more neater and much faster way of doing this?

    Code:
    FUNCTION SbsRemoveBlanks AS L (VarLine AS C)
    	dim VarHexcode as c = "0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F"
    	VarHexcode = comma_to_crlf(VarHexcode)
    	FOR i = 1 TO 6
    		IF is_one_of(substr(varLine,i,1),VarHexcode) THEN
    			SbsRemoveBlanks = .t.
    		ELSE
    			SbsRemoveBlanks = .f.
    			exit for
    		END IF
    	NEXT i
    END FUNCTION
    --
    Support your local Search and Rescue Unit, Get Lost!

    www.westrowops.co.uk

    #2
    Re: How can I make this function run quicker

    Originally posted by Graham Wickens View Post
    The following function checks the passed field [VarLine] to see if it has the chars/digits 0-9,A-F, ie its a 6 byte HexValue. It works but takes hours to run against 47000+ records.

    Is there a more neater and much faster way of doing this?

    Code:
    FUNCTION SbsRemoveBlanks AS L (VarLine AS C)
    	dim VarHexcode as c = "0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F"
    	VarHexcode = comma_to_crlf(VarHexcode)
    	FOR i = 1 TO 6
    		IF is_one_of(substr(varLine,i,1),VarHexcode) THEN
    			SbsRemoveBlanks = .t.
    		ELSE
    			SbsRemoveBlanks = .f.
    			exit for
    		END IF
    	NEXT i
    END FUNCTION
    The code looks ok. The purpose is to check the first six positions of a character string to see if they are valid in a 1-F sense? Possibly a different approach would be faster. I'd have to think about alternatives if my guess of the purpose is correct.
    There can be only one.

    Comment


      #3
      Re: How can I make this function run quicker

      I would try it with REGEX_MATCH().

      Take the first 6 characters and check those in a:

      ? regex_match("1f345A","[0-9,a-f]+","i")
      .T.

      ------------------- Added ---------------

      Code:
      FUNCTION SbsRemoveBlanks AS L (VarLine AS C)
         SbsRemoveBlanks = regex_match(left(varLine,6),"[0-9,a-f]+","i")
      END FUNCTION
      I even wouldn't make it a function. I would insert it right at the place where it is needed.

      OR

      Code:
      FUNCTION SbsRemoveBlanks AS L (VarLine AS C)
         SbsRemoveBlanks = regex_match(left(varLine,6),"[[:xdigit:]]+","i")
      END FUNCTION

      ------------------- Added 2 ----------------
      Code:
      FUNCTION SbsRemoveBlanks AS L (VarLine AS C)
      dim VarHexcode as c = "0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F"
      VarHexcode = comma_to_crlf(VarHexcode)
      
      SbsRemoveBlanks = .T.
      	
      FOR i = 1 TO 6
          IF .not. is_one_of(substr(varLine,i,1),VarHexcode) THEN
              SbsRemoveBlanks = .f.
              exit for
          END IF
      NEXT i
      END FUNCTION
      Last edited by Marcel Kollenaar; 02-25-2007, 06:55 PM. Reason: Had some better ideas and added it ...
      Marcel

      I hear and I forget. I see and I remember. I do and I understand.
      ---- Confusius ----

      Comment


        #4
        Re: How can I make this function run quicker

        Wow, thanks Marcel, that went like greased weasal droppings instead of the slow walk of my version. Thanks :)
        --
        Support your local Search and Rescue Unit, Get Lost!

        www.westrowops.co.uk

        Comment


          #5
          Re: How can I make this function run quicker

          Hi Marcel,

          how would I code the Regex to search the whole record for any valid hexcode?

          I tried "flag = regex_match(varLine,"[[:xdigit:]]+","i")" but it did not work, it gave me 12 records with just a number in each !!!:(
          --
          Support your local Search and Rescue Unit, Get Lost!

          www.westrowops.co.uk

          Comment


            #6
            Re: How can I make this function run quicker

            Regex is a powerful tool and everyone should no how to use it (In fact, it was a topic of one of the proposed talks I was going to give at the Alpha 5 2007 Conference this year, but that one was declined!). But it's sort of using a sledge hammer to push in a thumbtack. A simple push will do.

            The regex expression can be optimized a bit by returning .f. on the 1st non-match of a hex character.

            Another version, perhaps more simplistic (doesn't use regex) and runs a few percent faster than Marcel's

            Code:
            SbsRemoveBlanks = regex_match(left(varLine,6),"[[:xdigit:]]+","i")
            is below. It also potentially will run faster in a future version of Alpha (assuming a code compiler is ever created). The Regex is running from a compiled code DLL already, so it can never increase much in speed.

            Code:
            FUNCTION SbsRemoveBlanks AS A (VarLine AS C)
                SbsRemoveBlanks=IF(trim(chrtran(left(VarLine,6),"0123456789ABCDEF ","                :"))=="",.t.,.f.)
            END FUNCTION
            Regards,

            Ira J. Perlow
            Computer Systems Design


            CSDA A5 Products
            New - Free CSDA DiagInfo - v1.39, 30 Apr 2013
            CSDA Barcode Functions

            CSDA Code Utility
            CSDA Screen Capture


            Comment


              #7
              Re: How can I make this function run quicker

              Thanks Ira,

              you state:

              The regex expression can be optimized a bit by returning .f. on the 1st non-match of a hex character.
              so how do I do it!

              p.s.

              WHen I get a bit more competant with Regex, I have several Scripts to revisit that need speeding up!!
              Last edited by Graham Wickens; 02-27-2007, 02:42 PM. Reason: p.s.
              --
              Support your local Search and Rescue Unit, Get Lost!

              www.westrowops.co.uk

              Comment


                #8
                Re: How can I make this function run quicker

                Ok,

                I ran a comparison between the "regex_match" and "Remove Blanks" methods.

                from an input file of 47734 records

                the "regex_match" produced 33515 lines with a valid hexcode in the left 6 characters.

                the "Remove Blanks" produced 36664 lines with a valid hexcode in the left 6 characters, but include the lines without a valid hexcode as just cr/lf.

                Below is a sample of the records that I want to process (amongst a lot of other lines from the original emails).


                Code CallSign Reg ICAO FAlt LAlt Date/Time (dd/mm/yyyy)
                15409A -------- RA-82074.A124.-----.-----.26/02/2007.17:32:20
                3413CA.IBE4133.EC-HGZ.. A320.15425.21700.26/02/2007 12:55:40


                <tab>Code Callsign Reg Altitude Track Last Update
                <tab>341583/214.....EC-IDT.33,000.ft.203.6�.13:14:17
                <tab>400B8C.616......G-SAAW.36,000.ft.2.4�.15:07:31


                ......Code...Callsign........Reg......LastUpdate Altitude Track
                ......40075E BMI2972.........G-TOYG...22:47:18...15,000ft 263.7�
                ......400A15 EZY6420.........G-EZKA...22:47:18...38,000ft 2.9�
                ......400EFE EZY742..........G-EZBD...22:47:18...21,600ft 146.8�

                where dot = space chr(32)
                Code being the valid hexcode.

                the problem is made worse because the log format and columns differ from one log to the next, despite trying to get some conformity to allow for easier automated processing. The only fields of interest to me are the Hexcode, Registration, ICAO (TypeCode).
                Last edited by Graham Wickens; 02-27-2007, 04:36 PM. Reason: clarity
                --
                Support your local Search and Rescue Unit, Get Lost!

                www.westrowops.co.uk

                Comment


                  #9
                  Re: How can I make this function run quicker

                  Graham,

                  Is it an idea that before you merge the logs together, inspect them first with some 'intelligent' routines and format them in fixed space records and then process them?
                  Marcel

                  I hear and I forget. I see and I remember. I do and I understand.
                  ---- Confusius ----

                  Comment

                  Working...
                  X