Alpha Software Mobile Development Tools:   Alpha Anywhere    |   Alpha TransForm subscribe to our YouTube Channel  Follow Us on LinkedIn  Follow Us on Twitter  Follow Us on Facebook
Devcon 2020 Goes Virtual! LEARN MORE >

Announcement

Collapse

The Alpha Software Forum Participation Guidelines

The Alpha Software Forum is a free forum created for Alpha Software Developer Community to ask for help, exchange ideas, and share solutions. Alpha Software strives to create an environment where all members of the community can feel safe to participate. In order to ensure the Alpha Software Forum is a place where all feel welcome, forum participants are expected to behave as follows:
  • Be professional in your conduct
  • Be kind to others
  • Be constructive when giving feedback
  • Be open to new ideas and suggestions
  • Stay on topic


Be sure all comments and threads you post are respectful. Posts that contain any of the following content will be considered a violation of your agreement as a member of the Alpha Software Forum Community and will be moderated:
  • Spam.
  • Vulgar language.
  • Quotes from private conversations without permission, including pricing and other sales related discussions.
  • Personal attacks, insults, or subtle put-downs.
  • Harassment, bullying, threatening, mocking, shaming, or deriding anyone.
  • Sexist, racist, homophobic, transphobic, ableist, or otherwise discriminatory jokes and language.
  • Sexually explicit or violent material, links, or language.
  • Pirated, hacked, or copyright-infringing material.
  • Encouraging of others to engage in the above behaviors.


If a thread or post is found to contain any of the content outlined above, a moderator may choose to take one of the following actions:
  • Remove the Post or Thread - the content is removed from the forum.
  • Place the User in Moderation - all posts and new threads must be approved by a moderator before they are posted.
  • Temporarily Ban the User - user is banned from forum for a period of time.
  • Permanently Ban the User - user is permanently banned from the forum.


Moderators may also rename posts and threads if they are too generic or do not property reflect the content.

Moderators may move threads if they have been posted in the incorrect forum.

Threads/Posts questioning specific moderator decisions or actions (such as "why was a user banned?") are not allowed and will be removed.

The owners of Alpha Software Corporation (Forum Owner) reserve the right to remove, edit, move, or close any thread for any reason; or ban any forum member without notice, reason, or explanation.

Community members are encouraged to click the "Report Post" icon in the lower left of a given post if they feel the post is in violation of the rules. This will alert the Moderators to take a look.

Alpha Software Corporation may amend the guidelines from time to time and may also vary the procedures it sets out where appropriate in a particular case. Your agreement to comply with the guidelines will be deemed agreement to any changes to it.



Bonus TIPS for Successful Posting

Try a Search First
It is highly recommended that a Search be done on your topic before posting, as many questions have been answered in prior posts. As with any search engine, the shorter the search term, the more "hits" will be returned, but the more specific the search term is, the greater the relevance of those "hits". Searching for "table" might well return every message on the board while "tablesum" would greatly restrict the number of messages returned.

When you do post
First, make sure you are posting your question in the correct forum. For example, if you post an issue regarding Desktop applications on the Mobile & Browser Applications board , not only will your question not be seen by the appropriate audience, it may also be removed or relocated.

The more detail you provide about your problem or question, the more likely someone is to understand your request and be able to help. A sample database with a minimum of records (and its support files, zipped together) will make it much easier to diagnose issues with your application. Screen shots of error messages are especially helpful.

When explaining how to reproduce your problem, please be as detailed as possible. Describe every step, click-by-click and keypress-by-keypress. Otherwise when others try to duplicate your problem, they may do something slightly different and end up with different results.

A note about attachments
You may only attach one file to each message. Attachment file size is limited to 2MB. If you need to include several files, you may do so by zipping them into a single archive.

If you forgot to attach your files to your post, please do NOT create a new thread. Instead, reply to your original message and attach the file there.

When attaching screen shots, it is best to attach an image file (.BMP, .JPG, .GIF, .PNG, etc.) or a zip file of several images, as opposed to a Word document containing the screen shots. Because Word documents are prone to viruses, many message board users will not open your Word file, therefore limiting their ability to help you.

Similarly, if you are uploading a zipped archive, you should simply create a .ZIP file and not a self-extracting .EXE as many users will not run your EXE file.
See more
See less

Bulk import in a CSV format

Collapse
X
Collapse
 
  • Page of 1
  • Filter
  • Time
  • Show
Clear All
new posts
Previous template Next
    #1

    Bulk import in a CSV format


    Hello

    I am importing just over 400,000 records from a text file and it is incredibly slow is this normal. Data transfer is about 100 records ever 5 seconds

    John
    Failure makes improvements
    Tags: None
    #2
    RE: Bulk import in a CSV format

    Also

    Alpha tends to hang after about 8000 records

    JOhn
    Failure makes improvements

    Comment

      #3
      RE: Bulk import in a CSV format

      John, do you have an Excel "icon" for the csv files on your system?

      I am importing CSV files, but on the systems I am currently using, they have an Excel icon, and I am, via xbasic, open and saving them as Excel 4.0 and then importing them as Excel data. I don't know if this would be faster, but would be glad to share the code with you.
      Cole Custom Programming - Terrell, Texas
      972 524 8714
      [email protected]

      ____________________
      "A young man who is not liberal has no heart, but an old man who is not conservative has no mind." GB Shaw

      Comment

        #4
        RE: Bulk import in a CSV format

        John:

        It sounds like someyhing else is going on. Imports normally fly. The size of the input file should not have the impact you are experiencing.

        I import larger files very quickly using the import operation and xbasic.

        can you post the routine and a subset of the file.

        bob adler

        Comment

          #5
          RE: Bulk import in a CSV format

          Martin

          When you say an excel icon do you mean the program or something else. If it is excel the program I have it.

          John
          Failure makes improvements

          Comment

            #6
            RE: Bulk import in a CSV format

            What I mean is that the CSV files on the system where I get them display an Excel icon instead of a Text icon in the folders where they reside. I think in my case the reason for that is that the CSV files come from a web server where I think that the sending program saves them from an Excel "Web Page" as CSV, and display with Excel based icons.

            I would try Bob's suggestion, though, and post the routine, especially as he regularly imports files even larger.
            Cole Custom Programming - Terrell, Texas
            972 524 8714
            [email protected]

            ____________________
            "A young man who is not liberal has no heart, but an old man who is not conservative has no mind." GB Shaw

            Comment

              #7
              RE: Bulk import in a CSV format

              Hello Robert

              Here's the data base

              It�s a very simple structure I export to another database after the initial import.

              My problem lies with the initial import from the text document in CSV format

              John
              Failure makes improvements

              Comment

                #8
                RE: Bulk import in a CSV format

                John,

                Does the receiving table have any field rules which are being evaluated as each new record is entered to the table? -- tom

                Comment

                  #9
                  RE: Bulk import in a CSV format

                  Robert

                  When I have done this in the past all has gone well. I usually import around 100K at a time. This is my first attempt at 400K.

                  Now when I do the smaller import the same problem happens, very slow data transfer

                  John
                  Failure makes improvements

                  Comment

                    #10
                    RE: Bulk import in a CSV format

                    The speed you experience isn't normal. Usually, calculated or other field rules are the culprit. Just remove the field rules, run the import and then put 'em back.

                    - Steve
                    -Steve
                    sigpic

                    Comment

                      #11
                      RE: Bulk import in a CSV format

                      Tom

                      No it�s a simple table with no updating going on

                      All it does is imports from a CSV document into a table with 8 fields

                      No field rules no operations, nothing

                      John
                      Failure makes improvements

                      Comment

                        #12
                        RE: Bulk import in a CSV format

                        To all contributors

                        I created a new import operation and it worked flawlessly. 10 seconds process time for 400K records

                        Thanks for all input

                        John
                        Failure makes improvements

                        Comment

                        Previous template Next
                        Working...
                        X