Alpha Video Training
Results 1 to 9 of 9

Thread: How can I make this function run quicker

  1. #1
    Member Graham Wickens's Avatar
    Real Name
    Graham Wickens
    Join Date
    Apr 2000
    Location
    Gloucestershire, UK
    Posts
    795

    Default How can I make this function run quicker

    The following function checks the passed field [VarLine] to see if it has the chars/digits 0-9,A-F, ie its a 6 byte HexValue. It works but takes hours to run against 47000+ records.

    Is there a more neater and much faster way of doing this?

    Code:
    FUNCTION SbsRemoveBlanks AS L (VarLine AS C)
    	dim VarHexcode as c = "0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F"
    	VarHexcode = comma_to_crlf(VarHexcode)
    	FOR i = 1 TO 6
    		IF is_one_of(substr(varLine,i,1),VarHexcode) THEN
    			SbsRemoveBlanks = .t.
    		ELSE
    			SbsRemoveBlanks = .f.
    			exit for
    		END IF
    	NEXT i
    END FUNCTION
    --
    Support your local Search and Rescue Unit, Get Lost!

    www.westrowops.co.uk

  2. #2
    "Certified" Alphaholic Stan Mathews's Avatar
    Real Name
    Stan Mathews
    Join Date
    Apr 2000
    Location
    Bowling Green, KY
    Posts
    25,119

    Default Re: How can I make this function run quicker

    Quote Originally Posted by Graham Wickens View Post
    The following function checks the passed field [VarLine] to see if it has the chars/digits 0-9,A-F, ie its a 6 byte HexValue. It works but takes hours to run against 47000+ records.

    Is there a more neater and much faster way of doing this?

    Code:
    FUNCTION SbsRemoveBlanks AS L (VarLine AS C)
    	dim VarHexcode as c = "0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F"
    	VarHexcode = comma_to_crlf(VarHexcode)
    	FOR i = 1 TO 6
    		IF is_one_of(substr(varLine,i,1),VarHexcode) THEN
    			SbsRemoveBlanks = .t.
    		ELSE
    			SbsRemoveBlanks = .f.
    			exit for
    		END IF
    	NEXT i
    END FUNCTION
    The code looks ok. The purpose is to check the first six positions of a character string to see if they are valid in a 1-F sense? Possibly a different approach would be faster. I'd have to think about alternatives if my guess of the purpose is correct.

  3. #3
    "Certified" Alphaholic Marcel Kollenaar's Avatar
    Real Name
    Marcel
    Join Date
    Apr 2003
    Location
    Twente
    Posts
    1,514

    Default Re: How can I make this function run quicker

    I would try it with REGEX_MATCH().

    Take the first 6 characters and check those in a:

    ? regex_match("1f345A","[0-9,a-f]+","i")
    .T.

    ------------------- Added ---------------

    Code:
    FUNCTION SbsRemoveBlanks AS L (VarLine AS C)
       SbsRemoveBlanks = regex_match(left(varLine,6),"[0-9,a-f]+","i")
    END FUNCTION
    I even wouldn't make it a function. I would insert it right at the place where it is needed.

    OR

    Code:
    FUNCTION SbsRemoveBlanks AS L (VarLine AS C)
       SbsRemoveBlanks = regex_match(left(varLine,6),"[[:xdigit:]]+","i")
    END FUNCTION

    ------------------- Added 2 ----------------
    Code:
    FUNCTION SbsRemoveBlanks AS L (VarLine AS C)
    dim VarHexcode as c = "0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F"
    VarHexcode = comma_to_crlf(VarHexcode)
    
    SbsRemoveBlanks = .T.
    	
    FOR i = 1 TO 6
        IF .not. is_one_of(substr(varLine,i,1),VarHexcode) THEN
            SbsRemoveBlanks = .f.
            exit for
        END IF
    NEXT i
    END FUNCTION
    Last edited by Marcel Kollenaar; 02-25-2007 at 06:55 PM. Reason: Had some better ideas and added it ...
    Marcel

    I hear and I forget. I see and I remember. I do and I understand.
    ---- Confusius ----

  4. #4
    Member Graham Wickens's Avatar
    Real Name
    Graham Wickens
    Join Date
    Apr 2000
    Location
    Gloucestershire, UK
    Posts
    795

    Default Re: How can I make this function run quicker

    Wow, thanks Marcel, that went like greased weasal droppings instead of the slow walk of my version. Thanks :)
    --
    Support your local Search and Rescue Unit, Get Lost!

    www.westrowops.co.uk

  5. #5
    Member Graham Wickens's Avatar
    Real Name
    Graham Wickens
    Join Date
    Apr 2000
    Location
    Gloucestershire, UK
    Posts
    795

    Default Re: How can I make this function run quicker

    Hi Marcel,

    how would I code the Regex to search the whole record for any valid hexcode?

    I tried "flag = regex_match(varLine,"[[:xdigit:]]+","i")" but it did not work, it gave me 12 records with just a number in each !!!:(
    --
    Support your local Search and Rescue Unit, Get Lost!

    www.westrowops.co.uk

  6. #6
    VAR csda1's Avatar
    Real Name
    Ira J Perlow
    Join Date
    Apr 2000
    Location
    Boston, Massachusetts, USA
    Posts
    3,530

    Default Re: How can I make this function run quicker

    Regex is a powerful tool and everyone should no how to use it (In fact, it was a topic of one of the proposed talks I was going to give at the Alpha 5 2007 Conference this year, but that one was declined!). But it's sort of using a sledge hammer to push in a thumbtack. A simple push will do.

    The regex expression can be optimized a bit by returning .f. on the 1st non-match of a hex character.

    Another version, perhaps more simplistic (doesn't use regex) and runs a few percent faster than Marcel's

    Code:
    SbsRemoveBlanks = regex_match(left(varLine,6),"[[:xdigit:]]+","i")
    is below. It also potentially will run faster in a future version of Alpha (assuming a code compiler is ever created). The Regex is running from a compiled code DLL already, so it can never increase much in speed.

    Code:
    FUNCTION SbsRemoveBlanks AS A (VarLine AS C)
        SbsRemoveBlanks=IF(trim(chrtran(left(VarLine,6),"0123456789ABCDEF ","                :"))=="",.t.,.f.)
    END FUNCTION
    Regards,

    Ira J. Perlow
    Computer Systems Design


    CSDA A5 Products
    New - Free CSDA DiagInfo - v1.39, 30 Apr 2013
    CSDA Barcode Functions

    CSDA Code Utility
    CSDA Screen Capture



  7. #7
    Member Graham Wickens's Avatar
    Real Name
    Graham Wickens
    Join Date
    Apr 2000
    Location
    Gloucestershire, UK
    Posts
    795

    Default Re: How can I make this function run quicker

    Thanks Ira,

    you state:

    The regex expression can be optimized a bit by returning .f. on the 1st non-match of a hex character.
    so how do I do it!

    p.s.

    WHen I get a bit more competant with Regex, I have several Scripts to revisit that need speeding up!!
    Last edited by Graham Wickens; 02-27-2007 at 02:42 PM. Reason: p.s.
    --
    Support your local Search and Rescue Unit, Get Lost!

    www.westrowops.co.uk

  8. #8
    Member Graham Wickens's Avatar
    Real Name
    Graham Wickens
    Join Date
    Apr 2000
    Location
    Gloucestershire, UK
    Posts
    795

    Default Re: How can I make this function run quicker

    Ok,

    I ran a comparison between the "regex_match" and "Remove Blanks" methods.

    from an input file of 47734 records

    the "regex_match" produced 33515 lines with a valid hexcode in the left 6 characters.

    the "Remove Blanks" produced 36664 lines with a valid hexcode in the left 6 characters, but include the lines without a valid hexcode as just cr/lf.

    Below is a sample of the records that I want to process (amongst a lot of other lines from the original emails).


    Code CallSign Reg ICAO FAlt LAlt Date/Time (dd/mm/yyyy)
    15409A -------- RA-82074.A124.-----.-----.26/02/2007.17:32:20
    3413CA.IBE4133.EC-HGZ.. A320.15425.21700.26/02/2007 12:55:40


    <tab>Code Callsign Reg Altitude Track Last Update
    <tab>341583/214.....EC-IDT.33,000.ft.203.6°.13:14:17
    <tab>400B8C.616......G-SAAW.36,000.ft.2.4°.15:07:31


    ......Code...Callsign........Reg......LastUpdate Altitude Track
    ......40075E BMI2972.........G-TOYG...22:47:18...15,000ft 263.7°
    ......400A15 EZY6420.........G-EZKA...22:47:18...38,000ft 2.9°
    ......400EFE EZY742..........G-EZBD...22:47:18...21,600ft 146.8°

    where dot = space chr(32)
    Code being the valid hexcode.

    the problem is made worse because the log format and columns differ from one log to the next, despite trying to get some conformity to allow for easier automated processing. The only fields of interest to me are the Hexcode, Registration, ICAO (TypeCode).
    Last edited by Graham Wickens; 02-27-2007 at 04:36 PM. Reason: clarity
    --
    Support your local Search and Rescue Unit, Get Lost!

    www.westrowops.co.uk

  9. #9
    "Certified" Alphaholic Marcel Kollenaar's Avatar
    Real Name
    Marcel
    Join Date
    Apr 2003
    Location
    Twente
    Posts
    1,514

    Default Re: How can I make this function run quicker

    Graham,

    Is it an idea that before you merge the logs together, inspect them first with some 'intelligent' routines and format them in fixed space records and then process them?
    Marcel

    I hear and I forget. I see and I remember. I do and I understand.
    ---- Confusius ----

Similar Threads

  1. Can this routine be made quicker!
    By Chris.Tanti in forum Alpha Five Version 7
    Replies: 20
    Last Post: 11-24-2006, 01:59 PM
  2. How do you make a pointer paramter optional for a function?
    By mbmonk in forum Alpha Five Version 7
    Replies: 2
    Last Post: 02-24-2006, 05:09 PM
  3. Do Adhoc forms really load 20% quicker???
    By Mattyau in forum Alpha Five Version 6
    Replies: 2
    Last Post: 06-30-2005, 04:39 PM
  4. A4v7 report got quicker in W2000
    By Richard Middleton in forum Alpha Four Versions 7 and 8
    Replies: 1
    Last Post: 07-06-2003, 04:56 AM
  5. Is there a quicker way?
    By glenschild in forum Alpha Five Version 5
    Replies: 4
    Last Post: 08-06-2002, 09:42 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •