Alpha Software Mobile Development Tools:   Alpha Anywhere    |   Alpha TransForm subscribe to our YouTube Channel  Follow Us on LinkedIn  Follow Us on Twitter  Follow Us on Facebook

Announcement

Collapse

The Alpha Software Forum Participation Guidelines

The Alpha Software Forum is a free forum created for Alpha Software Developer Community to ask for help, exchange ideas, and share solutions. Alpha Software strives to create an environment where all members of the community can feel safe to participate. In order to ensure the Alpha Software Forum is a place where all feel welcome, forum participants are expected to behave as follows:
  • Be professional in your conduct
  • Be kind to others
  • Be constructive when giving feedback
  • Be open to new ideas and suggestions
  • Stay on topic


Be sure all comments and threads you post are respectful. Posts that contain any of the following content will be considered a violation of your agreement as a member of the Alpha Software Forum Community and will be moderated:
  • Spam.
  • Vulgar language.
  • Quotes from private conversations without permission, including pricing and other sales related discussions.
  • Personal attacks, insults, or subtle put-downs.
  • Harassment, bullying, threatening, mocking, shaming, or deriding anyone.
  • Sexist, racist, homophobic, transphobic, ableist, or otherwise discriminatory jokes and language.
  • Sexually explicit or violent material, links, or language.
  • Pirated, hacked, or copyright-infringing material.
  • Encouraging of others to engage in the above behaviors.


If a thread or post is found to contain any of the content outlined above, a moderator may choose to take one of the following actions:
  • Remove the Post or Thread - the content is removed from the forum.
  • Place the User in Moderation - all posts and new threads must be approved by a moderator before they are posted.
  • Temporarily Ban the User - user is banned from forum for a period of time.
  • Permanently Ban the User - user is permanently banned from the forum.


Moderators may also rename posts and threads if they are too generic or do not property reflect the content.

Moderators may move threads if they have been posted in the incorrect forum.

Threads/Posts questioning specific moderator decisions or actions (such as "why was a user banned?") are not allowed and will be removed.

The owners of Alpha Software Corporation (Forum Owner) reserve the right to remove, edit, move, or close any thread for any reason; or ban any forum member without notice, reason, or explanation.

Community members are encouraged to click the "Report Post" icon in the lower left of a given post if they feel the post is in violation of the rules. This will alert the Moderators to take a look.

Alpha Software Corporation may amend the guidelines from time to time and may also vary the procedures it sets out where appropriate in a particular case. Your agreement to comply with the guidelines will be deemed agreement to any changes to it.



Bonus TIPS for Successful Posting

Try a Search First
It is highly recommended that a Search be done on your topic before posting, as many questions have been answered in prior posts. As with any search engine, the shorter the search term, the more "hits" will be returned, but the more specific the search term is, the greater the relevance of those "hits". Searching for "table" might well return every message on the board while "tablesum" would greatly restrict the number of messages returned.

When you do post
First, make sure you are posting your question in the correct forum. For example, if you post an issue regarding Desktop applications on the Mobile & Browser Applications board , not only will your question not be seen by the appropriate audience, it may also be removed or relocated.

The more detail you provide about your problem or question, the more likely someone is to understand your request and be able to help. A sample database with a minimum of records (and its support files, zipped together) will make it much easier to diagnose issues with your application. Screen shots of error messages are especially helpful.

When explaining how to reproduce your problem, please be as detailed as possible. Describe every step, click-by-click and keypress-by-keypress. Otherwise when others try to duplicate your problem, they may do something slightly different and end up with different results.

A note about attachments
You may only attach one file to each message. Attachment file size is limited to 2MB. If you need to include several files, you may do so by zipping them into a single archive.

If you forgot to attach your files to your post, please do NOT create a new thread. Instead, reply to your original message and attach the file there.

When attaching screen shots, it is best to attach an image file (.BMP, .JPG, .GIF, .PNG, etc.) or a zip file of several images, as opposed to a Word document containing the screen shots. Because Word documents are prone to viruses, many message board users will not open your Word file, therefore limiting their ability to help you.

Similarly, if you are uploading a zipped archive, you should simply create a .ZIP file and not a self-extracting .EXE as many users will not run your EXE file.
See more
See less

corrupted indexes

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    #16
    RE: corrupted indexes

    It's not in a batch process. Basically the sequence is as follows:
    1) The new customer list in imported as a table.
    2) Starting at the begining, each record is checked for being a new customer.
    3) If the customer is new, a custbl.enter_begin() statement is used.
    4) The customer table (cus.dbf) fields are populated, and then a custbl.enter_end(.T.) statement is used.
    5) After every 100 new customers, the production indexes are updated, the table is closed, and then reopened.

    There is no batching. Adding the production indexes update and closing then reopening the table had no effect on the index corruption whatsoever.

    Comment


      #17
      RE: corrupted indexes

      Richard,

      If your sequence is running in a single script your troubles may stem from timing conflicts. Your script will not pause all by itself while the reindexing of each 100 records occurs. (Or while the new customer record is saved to disk). Generally, the script will continue running while input / output processes ( from / to disk) run asynchronously. When writing the script you may be assuming that the script will stop and wait for the disk i/o process to finish before continuing. This is often NOT the case. I suggest you liberally scatter a series of

      xbasic_wait_for_idle()

      statements throughout the script, to force the script to pause until each i/o sequence is finished.

      I'd use one after each and every statement that causes Alpha to write something to disk.

      -- tom

      Comment


        #18
        RE: corrupted indexes

        I assumed that the drive cache would handle it. That's the first reasonable explination (and possible ray of hope) I've heard so far. I'll try it and let you know.

        Thanks,
        Rick Smith

        Comment


          #19
          RE: corrupted indexes

          It didn't help. Same problem. No differance.

          Comment


            #20
            RE: corrupted indexes

            Richard,

            I'm sorry to hear that. Maybe it's time to let someone else look over your shoulder? How about posting a working copy of the database, or a model of it that illustrates the problem?

            -- tom

            Comment


              #21
              RE: corrupted indexes

              Here's the script:
              notes:
              1) status() is a routine that writes to a status line on the screen
              2) The "Fix**()" routines do standard checks and corrections.
              3) It ALWAYS gags on the line custbl.enter_end(.T.)


              'Date Created: 21-May-2004 03:28:55 PM
              'Last Updated: 27-May-2004 09:42:47 AM
              'Created By :
              'Updated By :
              ''XBasic
              '-----------------------------------------------------------------------------------------------------------------
              ' Copyright (C) 2004 by Richard D. Smith, P.E.
              ' ALL RIGHTS RESERVED
              '-----------------------------------------------------------------------------------------------------------------
              'Project: DDC
              'File: *** GMUnified ***
              '-----------------------------------------------------------------------------------------------------------------
              'NOTES:
              ' This processes the GM Unified data format. There are only customer names in this file- no VINs
              '-----------------------------------------------------------------------------------------------------------------

              dim GLOBAL Dealer_Id as c
              dim GLOBAL MainPath as C
              dim shared custbl as P 'opened customer table
              dim shared ziptbl as P


              '==========================================================================================================
              '--------------------------------------
              'do entry corrections
              '--------------------------------------
              Function FixStuff as l(rawtbl as P)
              dim tmp as c

              if len(rawtbl.fname)"0 'non-commercial
              rawtbl.lname= FixName(rawtbl.lname)
              rawtbl.fname= FixName(rawtbl.fname)
              rawtbl.mname= FixName(rawtbl.mname)
              end if
              rawtbl.addr= FixName(rawtbl.addr)
              rawtbl.city= FixName(rawtbl.city)
              rawtbl.state= FixName(rawtbl.state)
              if .NOT. ISALPHA(rawtbl.state)
              rawtbl.state=DState
              end if
              rawtbl.zip= alltrim(rawtbl.zip)
              if VAL(rawtbl.zip)"0
              rawtbl.zip=""
              end if
              '--------------------------------------
              'fix blank State field
              '--------------------------------------
              if LEN(ALLTRIM(rawtbl.state)) "2
              rawtbl.state= DState
              end if
              '--------------------------
              'check ZIP code
              '--------------------------
              if LEN(ALLTRIM(rawtbl.zip)) = 5 .AND. VAL(rawtbl.zip)"0
              rec = ziptbl.fetch_find(rawtbl.zip)
              if rec"0
              rawtbl.city= ziptbl.city
              end if
              exit function
              end if
              if LEN(ALLTRIM(rawtbl.zip)) " 5
              rawtbl.zip=LEFT(rawtbl.zip,5)
              rec = ziptbl.fetch_find(rawtbl.zip)
              if rec"0
              rawtbl.city= ziptbl.city
              end if
              exit function
              end if
              '--------------------------------------
              'zip code blank or too small if here
              '--------------------------------------
              if LEN(ALLTRIM(custbl.Cus_City))"5
              '--------------------------------------
              'no city, no zip code- bail out
              '--------------------------------------
              exit function
              end if
              ziptbl.index_primary_put("City")
              rec = ziptbl.fetch_find(rawtbl.city)
              if rec"0
              while rawtbl.city=ziptbl.city
              if rawtbl.state=ziptbl.state
              rawtbl.zip= ziptbl.zip
              exit function
              end if
              ziptbl.fetch_next()
              end while
              end if
              ziptbl.index_primary_put("Zip")
              rawtbl.zip="00000"
              End Function


              dim File_to_read as c
              dim code as n
              dim response as n
              dim newcnewc as n
              dim strttme as c
              dim RecordNum as n
              dim import_destination_table as c
              dim ct as n
              dim Cust_ID as n

              :A5.window_title = "DDC (C) 2004 by Richard D. Smith, P.E."
              if len(Dealer_ID)"3 'see if Dealer_ID set
              ui_msg_box(" *** BAD SETUP ***","Incorrect configuration", UI_OK + UI_ATTENTION_SYMBOL)
              stop
              end if
              status("GM Unified data file")
              WriteToLog("� ** GM Unified Data Load:",Dealer_ID)
              on error goto no_file
              code=UI_YES_NO_CANCEL+UI_QUESTION_SYMBOL+UI_SECOND_BUTTON_DEFAULT
              response = ui_msg_box("Question"," ADD GM Unified Data?", code)
              IF response = UI_YES_SELECTED THEN
              '--------------------------------------
              'Yes selected, get file name
              '--------------------------------------
              File_to_read = ui_get_file("Customer Data","Raw Data(*.TXT)|Capture(*.CAP)|ALL(*.*)","","X")
              dir_put(MainPath)
              IF File_to_read = "" THEN
              ui_beep(UI_QUESTION_BEEP) 'honk
              END 'no file tagged
              ELSE
              code=UI_YES_NO+UI_ATTENTION_SYMBOL+UI_SECOND_BUTTON_DEFAULT
              response = ui_msg_box(" VERIFY GM UNIFIED DATA FILE NAME",File_to_read, code)
              IF response != UI_YES_SELECTED THEN
              end 'user aborted
              ELSE
              dir_put(MainPath)
              '--------------------------------------
              'file chosen and confirmed when here
              '--------------------------------------
              strttme= time() 'get current time
              on error goto no_file 'error trap (CYA)
              status(strttme+" WORKING......")
              StatusBar.Enable()
              StatusBar.Show()

              import_destination_table = MainPath+"\raw_GMUnified.dbf"
              import_filename = File_to_read

              import.type = 0
              import.names = "none"
              import.file = import_filename
              import.db = import_destination_table
              import.skip_lines = 0
              import.skip_bytes = 0
              import.field_sep = ","
              import.record_sep = ""CR""LF""
              import.text_qualifier = ""
              import.record_len = 149
              import.rem_blanks = .F.
              import.escape_newlines = .F.
              import.fields = 27
              import.field1 = "field_1,c,0,2,0"
              import.field2 = "zip,c,1,5,0"
              import.field3 = "dash,c,2,1,0"
              import.field4 = "zipp4,c,3,4,0" 'zip+4
              import.field5 = "field_5,c,4,3,0" 'delivery point code
              import.field6 = "lname,c,5,15,0"
              import.field7 = "fname,c,6,11,0"
              import.field8 = "mname,c,7,1,0"
              import.field9 = "suffix,c,8,2,0"
              import.field10 = "field_10,n,9,1,0" 'salut code (no decode available)
              import.field11 = "addr,c,10,34,0"
              import.field12 = "city,c,11,13,0"
              import.field13 = "state,c,12,2,0"
              import.field14 = "field_14,c,13,3,0" 'country code
              import.field15 = "field_15,n,14,3,0" 'assigned dealer manufacturer code
              import.field16 = "field_16,n,15,1,0" 'assigned dealer division code
              import.field17 = "field_17,c,16,2,0" 'GM vehicle group code
              import.field18 = "year,c,17,4,0"
              import.field19 = "field_19,c,18,2,0" 'vehicle subgroup code
              import.field20 = "field_20,c,19,2,0" 'vehicle series code
              import.field21 = "areacode,n,20,3,0"
              import.field22 = "phone,n,21,7,0"
              import.field23 = "field_23,c,22,1,0" 'blank
              import.field24 = "field_24,c,23,4,0" 'consumer carier route
              import.field25 = "model,c,24,16,0"
              import.field26 = "new_used,c,25,1,0"
              import.field27 = "field_27,c,26,1,0" 'blank
              import()

              'Add the import destination table to the database....
              file_add_to_db(import_destination_table)

              '--------------------------------------
              'show busy screen
              '--------------------------------------
              :form.view("BUSY")
              :BUSY.maximize()
              :BUSY:text2.text=File_to_read
              CheckIndexs()
              dir_put(MainPath)
              file.rename(File_to_read,File_to_read+".dne") 'change file name

              custbl=table.open(MainPath+"\cus.dbf") 'open customer table
              custbl.update_production_index()
              rawtbl=table.open(MainPath+"\raw_GMUnified.dbf") 'open raw data table
              ziptbl=table.open(MainPath+"\zipcode.dbf") 'open zip table
              ziptbl.index_primary_put("Zip")
              'start reading table
              mycount=rawtbl.records_get()
              status("Data correction process:"+ STR(INT(mycount))) 'Displays record number on status bar
              rawtbl.fetch_first()
              WHILE .NOT. rawtbl.fetch_eof()
              if MOD(mycount,50)=0
              status("Data correction process:"+ STR(INT(mycount))) 'Displays record number on status bar
              end if
              mycount=mycount-1
              if LEN(ALLTRIM(rawtbl.lname))"0 'non-blank
              rawtbl.change_begin()
              FixStuff(rawtbl)
              rawtbl.change_end(.T.)
              end if
              rawtbl.fetch_next()
              end while
              ziptbl.close()
              rawtbl.close()

              rawtbl=table.open(MainPath+"\raw_GMUnified.dbf") 'reopen raw data table
              rawtbl.fetch_first()
              newcnewc=0
              mycount=0
              status("GM Unified Customer Data process:"+ STR(INT(mycount)) + ": new cus="+str(newcnewc)) 'Displays record number on status bar
              WHILE .NOT. rawtbl.fetch_eof()
              if MOD(mycount,20)=0
              status("Unified Customer Data process:"+ STR(INT(mycount)) + ": new cus="+str(newcnewc)) 'Displays record number on status bar
              end if
              if LEN(ALLTRIM(rawtbl.lname))"0 'only non-blank entries
              mycount=mycount+1
              on error goto no_file '********error trap
              '---------------------------------------------
              'see if it's a new customer or just a new car
              '---------------------------------------------
              custbl.index_primary_put("CUS_LAST")
              rec_count= custbl.fetch_find(upper(alltrim(rawtbl.lname)))
              Cust_ID=-1 'init
              if rec_count "0 'last customer name found
              'check last name, first name, middle initial, and zip for existing customer
              while SMATCH(upper(ALLTRIM(custbl.CUS_LAST)), ALLTRIM(rawtbl.lname)) .AND. (.NOT. custbl.fetch_EOF() .AND. (Cust_ID"0))
              if len(rawtbl.fname)"0
              if SMATCH(upper(ALLTRIM(custbl.CUS_FIRST)),ALLTRIM(rawtbl.fname)) .and. SMATCH(upper(left(ALLTRIM(custbl.CUS_Middle),1)),ALLTRIM(rawtbl.mname)) .and. SMATCH(ALLTRIM(custbl.CUS_ZIP),ALLTRIM(rawtbl.ZIP))
              Cust_ID=custbl.Cust_ID
              end if
              else 'commercial
              if SMATCH(ALLTRIM(custbl.CUS_ZIP),ALLTRIM(rawtbl.ZIP))
              Cust_ID=custbl.Cust_ID
              end if
              end if
              custbl.fetch_next()
              end while
              end if
              if Cust_ID"0 'still init
              newcnewc=newcnewc+1
              '---------------------------------------------
              'new customer if here
              '---------------------------------------------
              custbl.index_primary_put("CUST_ID")
              'Get highest number- may be higher than number of records (from dup removal)
              custbl.fetch_last() ''''''''''''''
              Cust_ID= custbl.Cust_ID + 1 'next ID
              'safety check for Cust_ID
              while custbl.fetch_find(Cust_ID)"0 'already exists
              Cust_ID = Cust_ID + 1 'next ID
              end while
              WriteToLog("new CUST_ID from RO:"+STR(Cust_ID)+" :", Dealer_ID)
              '---------------------------
              'change customer table data
              '---------------------------
              custbl.enter_begin()
              custbl.Creation=Date()
              custbl.Namesource="GM Unified Data"
              custbl.status=1 'new customer, car purchase
              custbl.Cust_Id= Cust_ID
              custbl.Dealer_Id= Dealer_Id
              custbl.Cus_Prefix= "" '************don't have decode
              custbl.Cus_Last= upper(alltrim(rawtbl.lname))
              custbl.Cus_First= upper(rawtbl.fname)
              custbl.Cus_Middle= upper(rawtbl.mname)
              custbl.Cus_Suffix= rawtbl.suffix
              custbl.Cus_Street= rawtbl.addr
              custbl.Cus_Stree0= ""
              custbl.Cus_City= upper(rawtbl.CITY)
              custbl.Cus_State= upper(rawtbl.STATE)
              custbl.Cus_Zip= rawtbl.ZIP
              custbl.ZipDist= GetDistance(rawtbl.Zip) 'distance from dealer
              custbl.Cus_Home_P= (rawtbl.areacode * 100000) + rawtbl.phone
              custbl.Cus_Work_P= 0
              custbl.Cus_Cell_P= 0
              custbl.Cus_Email= ""
              custbl.Notify_Block_Flag=0 'no restrictions
              custbl.Notify_Preferance=0
              custbl.Cli=0
              custbl.Commercial=IsCommercial(rawtbl.fname, rawtbl.mname, rawtbl.lname, rawtbl.suffix, custbl.Commercial)
              custbl.enter_end(.T.)
              xbasic_wait_for_idle()
              custbl.index_primary_put("CUS_LAST")
              else 'old customer
              WriteToLog("Existing CUST_ID (no mods) :"+STR(Cust_ID)+" DLR:", Dealer_ID)
              end if 'still init
              '----------------------------------------------------------------
              'customer related stuff done
              '----------------------------------------------------------------
              end if 'if LEN(ALLTRIM(rawtbl.lname))"0
              rawtbl.fetch_next()
              END WHILE ''''''''' .NOT. fptr.EOF()
              status("index update....")
              custbl.index_create_begin("CUST_ID", "CUST_ID")
              custbl.index_add("CUS_LAST","CUS_LAST")
              index_pointer = custbl.index_create_end()
              custbl.close()
              rawtbl.close()
              :BUSY.close()
              Analized=0
              status("GM Unified Customer Data process:"+ STR(INT(mycount)) + ": new cus="+str(newcnewc)) 'Displays record number on status bar
              response = ui_msg_box(" DONE PROCESSING","** Done Processing GM Unified Customer Data File **", UI_OK + UI_ATTENTION_SYMBOL)
              END IF 'response != UI_YES_SELECTED
              END IF 'File_to_read = ""
              ELSE '''''''IF response = UI_NO_SELECTED THEN
              'No selected
              ui_beep(UI_QUESTION_BEEP) 'honk
              END
              END IF 'response = UI_YES_SELECTED

              END

              no_file:
              ErrorHandler()
              END

              Comment


                #22
                RE: corrupted indexes

                Richard,

                That's more than I can digest without dedicating a chunk of time to it. However, I still think you may have a timing issues in lots of places throughout the code. The import, the cleanup of the rawtable, the insertion of new customer records... you write things to disk frequently. I'd go back through the entire script, top to bottom, and pause the script each time you ask Alpha to read from or write to the disk.

                -- tom

                Comment


                  #23
                  RE: corrupted indexes

                  There seems to be a fairly consistant pattern. It gags after adding about 300 new records.

                  Comment


                    #24
                    RE: corrupted indexes

                    Richard,

                    I always open a table with exclusive read/write privileges before doing a reindex. Can't see that in your script, am I missing something?

                    -- tom

                    Comment


                      #25
                      RE: corrupted indexes

                      This is a single user application. I didn't bother with exclusivity because there is no one else or no other process using the table. The reindexing works every time. It always gags on the Enter_End(.T.) statement.

                      Comment


                        #26
                        RE: corrupted indexes

                        Richard,

                        I fear I'm not making my point. If your script is opening and closing the same table that's being indexed and reindexed, there ARE multiple processes going on at once any one of which could have the table locked. The script does not run synchronously.

                        Reindexing writes to disk. I'd pause the script while it goes on.

                        Saving a record to disk writes to disk. I'd pause the script while it occurs.

                        etc. and so on.

                        -- tom

                        Comment


                          #27
                          RE: corrupted indexes

                          Maybe it's my use of the word 'synchronously'.

                          Substitute 'sequentially' and see if that makes things clearer.

                          -- tom

                          Comment


                            #28
                            RE: corrupted indexes

                            I changed ALL table open statements to open FILE_RW_EXCLUSIVE. No effect. Same problem.

                            Comment


                              #29
                              RE: corrupted indexes

                              Try changing the definition of the Cust_Last index to, for example, Cust_Last+Cust_First. It would test my theory at least as it applies to that index.

                              F.
                              Finian

                              Comment


                                #30
                                RE: corrupted indexes

                                Changing the index definition from "cus_last" to "cus_last+cus_first" appears to have "solved" the problem, however, I can't leave the definition that way. Whatever the problem is, now that you know what it is, how do I get around it?

                                Rick Smith

                                Comment

                                Working...
                                X