Alpha Software Mobile Development Tools:   Alpha Anywhere    |   Alpha TransForm subscribe to our YouTube Channel  Follow Us on LinkedIn  Follow Us on Twitter  Follow Us on Facebook
Devcon 2020 Goes Virtual! LEARN MORE >

Announcement

Collapse

The Alpha Software Forum Participation Guidelines

The Alpha Software Forum is a free forum created for Alpha Software Developer Community to ask for help, exchange ideas, and share solutions. Alpha Software strives to create an environment where all members of the community can feel safe to participate. In order to ensure the Alpha Software Forum is a place where all feel welcome, forum participants are expected to behave as follows:
  • Be professional in your conduct
  • Be kind to others
  • Be constructive when giving feedback
  • Be open to new ideas and suggestions
  • Stay on topic


Be sure all comments and threads you post are respectful. Posts that contain any of the following content will be considered a violation of your agreement as a member of the Alpha Software Forum Community and will be moderated:
  • Spam.
  • Vulgar language.
  • Quotes from private conversations without permission, including pricing and other sales related discussions.
  • Personal attacks, insults, or subtle put-downs.
  • Harassment, bullying, threatening, mocking, shaming, or deriding anyone.
  • Sexist, racist, homophobic, transphobic, ableist, or otherwise discriminatory jokes and language.
  • Sexually explicit or violent material, links, or language.
  • Pirated, hacked, or copyright-infringing material.
  • Encouraging of others to engage in the above behaviors.


If a thread or post is found to contain any of the content outlined above, a moderator may choose to take one of the following actions:
  • Remove the Post or Thread - the content is removed from the forum.
  • Place the User in Moderation - all posts and new threads must be approved by a moderator before they are posted.
  • Temporarily Ban the User - user is banned from forum for a period of time.
  • Permanently Ban the User - user is permanently banned from the forum.


Moderators may also rename posts and threads if they are too generic or do not property reflect the content.

Moderators may move threads if they have been posted in the incorrect forum.

Threads/Posts questioning specific moderator decisions or actions (such as "why was a user banned?") are not allowed and will be removed.

The owners of Alpha Software Corporation (Forum Owner) reserve the right to remove, edit, move, or close any thread for any reason; or ban any forum member without notice, reason, or explanation.

Community members are encouraged to click the "Report Post" icon in the lower left of a given post if they feel the post is in violation of the rules. This will alert the Moderators to take a look.

Alpha Software Corporation may amend the guidelines from time to time and may also vary the procedures it sets out where appropriate in a particular case. Your agreement to comply with the guidelines will be deemed agreement to any changes to it.



Bonus TIPS for Successful Posting

Try a Search First
It is highly recommended that a Search be done on your topic before posting, as many questions have been answered in prior posts. As with any search engine, the shorter the search term, the more "hits" will be returned, but the more specific the search term is, the greater the relevance of those "hits". Searching for "table" might well return every message on the board while "tablesum" would greatly restrict the number of messages returned.

When you do post
First, make sure you are posting your question in the correct forum. For example, if you post an issue regarding Desktop applications on the Mobile & Browser Applications board , not only will your question not be seen by the appropriate audience, it may also be removed or relocated.

The more detail you provide about your problem or question, the more likely someone is to understand your request and be able to help. A sample database with a minimum of records (and its support files, zipped together) will make it much easier to diagnose issues with your application. Screen shots of error messages are especially helpful.

When explaining how to reproduce your problem, please be as detailed as possible. Describe every step, click-by-click and keypress-by-keypress. Otherwise when others try to duplicate your problem, they may do something slightly different and end up with different results.

A note about attachments
You may only attach one file to each message. Attachment file size is limited to 2MB. If you need to include several files, you may do so by zipping them into a single archive.

If you forgot to attach your files to your post, please do NOT create a new thread. Instead, reply to your original message and attach the file there.

When attaching screen shots, it is best to attach an image file (.BMP, .JPG, .GIF, .PNG, etc.) or a zip file of several images, as opposed to a Word document containing the screen shots. Because Word documents are prone to viruses, many message board users will not open your Word file, therefore limiting their ability to help you.

Similarly, if you are uploading a zipped archive, you should simply create a .ZIP file and not a self-extracting .EXE as many users will not run your EXE file.
See more
See less

Import and append speed is too slow...

Collapse
X
Collapse
 
  • Page of 1
  • Filter
  • Time
  • Show
Clear All
new posts
Previous template Next
    #1

    Import and append speed is too slow...

    Hey Guys,
    I'm currently importing data using an xbasic script. The data is imported from an Access .mdb file. Here is the process:

    1. *.mdb file is selected using a ui_msg box.
    2. The *.mdb data is imported into several temp tables.
    3. The temp tables are then appended to the current database. (Each table is either being updated using a "unique" search or a "unique, replace" search.

    Problem:It is just too darn slow!

    The import file size is under 10 megs in the Access format. Further down the road, some of these data files are going to be over 1 gig.

    Any thoughts to speeding up my processing? Could it be the table indexing? Could it be my coding? I'm attaching a text file with my scrpt on it if anyone want to take a stab at it. Any thoughts would be greatly appreciated!

    Steve Hall
    Steve Hall
    Tags: None
    #2
    Changing to

    Code:
    a_tbl = table.open("Cdr",FILE_RW_EXCLUSIVE)
    etcetera, should help.

    Many suggest not reusing pointers as you have with
    • a_tbl = table.open("Cdr")
    • a_tbl = table.open("phone")
    • a_tbl = table.open("service")

    but I don't imagine that contributes to the speed issue.
    There can be only one.

    Comment

      #3
      That worked.

      Stan,
      Yeah, that made a little difference....:) Any ideas as to why that speeds up the process that much?
      Steve Hall

      Comment

        #4
        Originally posted by spain246
        Stan,
        Yeah, that made a little difference....:) Any ideas as to why that speeds up the process that much?
        If I understand correctly, Alpha doesn't waste any processor cycles checking to see if any other users (whether there are any or not it still checks) needing access to the table.

        Do you have an index on the field(s) in the master table which determine what a unique record is? I think that can help as well.

        If you want to test the speed difference with and without the exclusive access you can wrap the operation code with

        Code:
        st = now()
[B][COLOR="DarkOrange"]operation code here[/COLOR][/B]
et = now()
ui_msg_box("Result","Processed in "+str(et-st)+" seconds.") seconds")
        There can be only one.

        Comment

          #5
          Unique Indexes

          Stan,
          Can you look at this picture and tell me if you think my index for this table is too cluttered? Adding the FILE_RW_EXCLUSIVE sped up the process considerably, but now when I import the "trans" table, it seems to be omitting a bunch of records. The master and append keys are set in the xbasic script (date, time, serial). The records in the original database don't have any duplicates with this combination, but it's still missing some files.


          Thanks!
          Steve Hall

          Comment

            #6
            Try downloading the trial version of A5v7 and use the ADO Import Genie. See if that is any quicker.
            Aaron Brown
            Alpha Software Development Team

            Comment

              #7
              Observation only

              I've never found a need for filtered indexes. I would experiment with three single expression indexes, one each for serial, date, and time. Then when the need arises for the filter, apply it in the filtering for the current form, browse, etc.

              I can't speak to your application but unless the users are constantly changing indexes to use those you have explicitly defined, there is a lot of unnecessary index maintenance required of Alpha by those definitions.
              There can be only one.

              Comment

                #8
                Unique indexes

                Stan,
                Would the indexes affect the append? Right now i'm only able to import 6515 records out of 9499. There are no duplicates.
                Steve Hall

                Comment

                  #9
                  Nevermind

                  Nevermind.

                  I just changed the order on the key and it seems to have fixed the problem.
                  Steve Hall

                  Comment

                  Previous template Next
                  Working...
                  X