Coordinating Development Files with Live Production Files

[Ted Giles]

Re: Coordinating Development Files with Live Production Files



Careful. Lenny is watching!

[Tom Henkel]

Re: Coordinating Development Files with Live Production Files

He's too young to remember Woodstock...

[CALocklin]

Re: Coordinating Development Files with Live Production Files

Sam, you've got the basics down pretty good. And the comments about indexes seem to have taken you in the right direction on that issue also.

The one thing I'd disagree with you about is the idea that you would need to make structure changes directly on the production DBFs. While this is true from a "strictly speaking" standpoint, I have to guess that what you really mean by that is that you will have to either go to the customer location (physically or remotely) and change the structure on their system or get a copy of their data, modify the table(s), and send it back - this should VERY seldom be the case. I haven't done that in years and I've made a lot of structure changes on existing applications.

Go to this thread for a full description of how I do it. There are also 4-5 more links in that response to other responses that will add even more detail. (In other words, this has been discussed a few times before - but it can be rather difficult to find those threads if you don't know what keyword to use. I used A5_get_fieldnames.)

Most of the discussion is about adding new fields because that's the most common situation. However, at least one of those links should show an example of how I use a5_changefieldsize() to change field width or the number of decimals.

Also, I know that I once created a function to "change the field name" but it was long time ago and I don't know where it is right now. (Or "if" it is for that matter.) I think it basically consisted of xbasic to create the new table, append the data, delete the old table, and rename the new table. Something similar could probably be done to change the field type. However, my current philosophy is: Don't do it - if the name is that bad or the field type is wrong, add a new field. Adding a new field is easy and, if you're an old fart like me, don't forget that with today's hard drives there's really not much need to worry about hard drive space like there used to be. Furthermore, there should never be a real need ("desire" maybe yes; "need" no) to change a field name because the user should never see the actual field name. You can call the field Cust_Name and put the invoice total in it. It might confuse you a bit 6 months down the road when debugging something but, other than that, it won't hurt anything.

And the idea of changing a field from character to number falls in the same basic catergory as far as I'm concerned - just add a new field and, if appropriate, populate it based on the values in the old field. Consider this, too - you are likely to destroy the data if you simply change the field type of an existing field. That means you will probably have to use the old data to update the new field. So, just create the new field, run some kind of update (I'd do it with xbasic) to initialize the existing records, and just ignore the old field. (At least one of those links should also explain why I never delete fields - it has to do with restoring old backups.) Another reason to keep the old data (BTDT) is that your update might go wrong somewhere and this way you will still have the old data if you need to fix it. Or, (been there, done this one too) you may have a problem that your customer blames on your update and having the old data may allow you to prove them wrong. (Or it may prove you wrong but at least you'll know where the problem was.)

FYI: Part of the reason for my philosophy is based on some generic apps that have been sold to multiple customers and some of those customers are using older versions than some other customers. That means that the fields that need to be added/updated are different for some people than for others. It also means that I definitely do NOT want to call each of those customers to get their data, edit it, and send it back - that's WAY more work than I want to do. If it was a "one customer" database, I'd consider a "manual" update. But never on a generic app and usually not even for "one customer" databases. (If you read the link(s) carefully you'll know why I seldom do manual updates even on the "one customer" databases. Hint - restoring backups.)

Some other developers do something like send new tables with the correct structure, append the old data, delete the old table, and rename the new table. My personal opinion is that that method is generally too cumbersome - especially for just adding new fields (which, at least for me, is 99% of the problem) but to each his own. It also negatively affects the "restoring old backups" issue.

[sgerber]

Re: Coordinating Development Files with Live Production Files

Sam, you've got the basics down pretty good. And the comments about indexes seem to have taken you in the right direction on that issue also.

The one thing I'd disagree with you about is the idea that you would need to make structure changes directly on the production DBFs. While this is true from a "strictly speaking" standpoint, I have to guess that what you really mean by that is that you will have to either go to the customer location (physically or remotely) and change the structure on their system or get a copy of their data, modify the table(s), and send it back - this should VERY seldom be the case. I haven't done that in years and I've made a lot of structure changes on existing applications.

Go to this thread for a full description of how I do it.

Cal, thanks for your post. You're the first person to confirm my understanding of the various Alpha file types. Stan's post about the CDX files was very helpful but he didn't actually say that I would now have a complete understanding.

Your comments about how you handle the automation of updating table fields are interesting. That seems essential if you have a clientele of multiple users of an application package or a single client with many users.

I would adopt the "manual" approach simply because changes in my son's environment are quite rare. In the course of upgrading his 16 year old Alpha app, we discovered a numeric field that was about to reach capacity and needed enlarging. Otherwise, Cal, there are two fields that will be put to new uses and I want to give them more appropriate names simply because I'm anal about things like that.

I probably will make these changes to the live tables purely for expediency. They're simple ones and I'm not interrupting a multi-user work flow.

Thanks for your interest and for sharing the ideas you've implemented.

Regards ... Sam

[draganksg]

Re: Coordinating Development Files with Live Production Files

Hi Sam, I'll share my own experience with you.

I learned a years ago that if I do not have any mathematical calculations over the field, this field is always best to be a character type.
I do not use auto increment , instead I have table that keeps my keys for every table that I need. End when I enter new record , I increase the value of that field.e

On client place I do the necessary changes to the structure of the fields in the tables "manualy".
In all my application I have a "maintenance form" protected by password. Among other things, I have a couple of buttons behinde which is the code for table maintenance.


1.) Edit index


dim cTableName as C
cTableName = ui_get_table("Tabele", "Table", "", .t., .t., .f.)

a5_edit_indexes(cTableName)

2.) Check index

a5_checkindexesdialog()


3.) Edit structure

dim cTableName as C
cTableName = ui_get_table("Tabele", "Table", "", .t., .t., .f.)

a5_edit_structure(cTableName)

4.) Edit data in table by default browse

dim cTableName as C
cTableName = ui_get_table("Tabele", "Table", "", .t., .t., .f.)

a5_open_default_browse(cTableName)

5.) And some table maintenance with code that I copy from this forum ( I don't remember who posted it I apologize for that)
FUNCTION clean_indexes AS L ( )

dim user_data.foo as C
user_data.foo = ""
dim cont as C
dim rslt_cont as C
dim cont_ln as C
dim ln as C

:registry.load_settings("Tables",User_Data)
cont = property_to_string(user_data)

for each fooc in cont
SELECT
CASE "<DBF" $ fooc
ln = stritran(fooc,"<DBF",".DBF")
cont_ln = extract_string(ln,"<","<",1)
rslt_cont = rslt_cont + cont_ln + crlf()

CASE "<ALB" $ fooc
ln = stritran(fooc,"<ALB",".ALB")
cont_ln = extract_string(ln,"<","<",1)
rslt_cont = rslt_cont + cont_ln + crlf()

CASE "<SET" $ fooc
ln = stritran(fooc,"<SET",".SET")
cont_ln = extract_string(ln,"<","<",1)
rslt_cont = rslt_cont + cont_ln + crlf()

CASE "<DDD" $ fooc
ln = stritran(fooc,"<DDD",".DDD")
cont_ln = extract_string(ln,"<","<",1)
rslt_cont = rslt_cont + cont_ln + crlf()

END SELECT
next

for each foox in rslt_cont
registry.drop_settings("Tables\\"+foox)
next

registry.drop_settings("Tables")

END FUNCTION

*********************
FUNCTION clean_indexes AS L ( )

dim user_data.foo as C
user_data.foo = ""
dim cont as C
dim rslt_cont as C
dim cont_ln as C
dim ln as C

:registry.load_settings("INDX",User_Data)
cont = property_to_string(user_data)
rslt_cont = ""
cont_ln = ""

FOR each fooc in cont
SELECT
CASE "<DBF" $ fooc
IF "-" $ fooc
keyn = extract_string(fooc,"<","<",1)
tblno = extract_string(fooc,";","<",1)
ln = strtran(fooc,"<DBF",".DBF")
cont_ln = extract_string(ln,"<","<",1)
rslt_cont = rslt_cont + cont_ln + crlf()
ELSE
tblno = increment_value(tblno)
cont_ln = keyn+".dbf"+";"+tblno
rslt_cont = rslt_cont + cont_ln + crlf()
END IF

CASE "<ALB" $ fooc
IF "-" $ fooc
keyn = extract_string(fooc,"<","<",1)
tblno = extract_string(fooc,";","<",1)
ln = strtran(fooc,"<ALB",".ALB")
cont_ln = extract_string(ln,"<","<",1)
rslt_cont = rslt_cont + cont_ln + crlf()
ELSE
tblno = increment_value(tblno)
cont_ln = keyn+".alb"+";"+tblno
rslt_cont = rslt_cont + cont_ln + crlf()
END IF

CASE "<SET" $ fooc
IF "-" $ fooc
keyn = extract_string(fooc,"<","<",1)
tblno = extract_string(fooc,";","<",1)
ln = strtran(fooc,"<SET",".SET")
cont_ln = extract_string(ln,"<","<",1)
rslt_cont = rslt_cont + cont_ln + crlf()
ELSE
tblno = increment_value(tblno)
cont_ln = keyn+".set"+";"+tblno
rslt_cont = rslt_cont + cont_ln + crlf()
END IF

CASE "<DDD" $ fooc
IF "-" $ fooc
keyn = extract_string(fooc,"<","<",1)
tblno = extract_string(fooc,";","<",1)
ln = strtran(fooc,"<DDD",".DDD")
cont_ln = extract_string(ln,"<","<",1)
rslt_cont = rslt_cont + cont_ln + crlf()
ELSE
tblno = increment_value(tblno)
cont_ln = keyn+".ddd"+";"+tblno
rslt_cont = rslt_cont + cont_ln + crlf()
END IF

END SELECT
next

FOR each foox in rslt_cont
registry.drop_settings("INDX\\"+foox)
next

registry.drop_settings("INDX")

END FUNCTION

**************
FUNCTION cleanit AS L ( )

DIM path[10] as c
path[1] = a5.get_path() + chr(92)
path[2] = a5.get_private_path() + chr(92)
pcount = 2

IF a5.get_private_path() <> a5.get_exe_path()
path[3] = a5.get_exe_path() + chr(92)
pcount = 3
END IF

fcount = 0
FOR x = 1 to pcount
'type_of_files = 0 + FILE_FIND_ARCHIVE + FILE_FIND_READONLY + FILE_FIND_HIDDEN
'dfiles = filefind.first( path[x] + "$$*.*", type_of_files )
dfiles = filefind.first( path[x] + "$$*.*", FILE_FIND_NOT_DIRECTORY )

WHILE .not.dfiles.eof()
fname = dfiles.name() '*** GET THE FILE NAME.
ON ERROR goto Continue_next '*** FIND OUT IF THE FILE IS LOCKED.
fp = file.open( fname, file_rw_exclusive )
ON ERROR goto 0

fp.close() '*** FILE IS NOT LOCKED SO CLOSE IT AND REMOVE IT.
file.remove( fname )
fcount = fcount + 1

'---------------
Continue_next:
'---------------
ON ERROR goto 0
dfiles.next() '*** GET NEXT FILE IN LIST.
statusbar.robot()
END WHILE
NEXT

END FUNCTION


There's more things relating to the maintenance, but depending on the type of application.

[sgerber]

Re: Coordinating Development Files with Live Production Files

Dragan, thanks for taking the time to post some of the techniques you've used. Fortunately my situation is not all that complicated since I don't have to automate any maintenance. My purpose in starting this thread was to make sure I understood the various Alpha Five file types well enough to change the appropriate files whenever I changed a form or report or something that didn't directly involve the data.

I'm at a point now where I think I can manually apply maintenance to our application without regressing the data tables.

Thanks again for your interest and for sharing your technique(s).

Regards ... Sam

[CALocklin]

Re: Coordinating Development Files with Live Production Files

Just to drag this out a little more...

In my previous post I mentioned the issue of backups. If you are doing "manual" restructuring and there is any possibility that your user might need to restore an old backup for some reason, your best bet is to make minimal changes to existing fields and put new fields at the end of the table. That may not ensure that the old files will still work in the updated app but it will probably work AND it's almost a guarantee that something will NOT work if you insert new fields in the middle of the table.

And be very careful about deciding that an old backup will not be needed. I've had customers who didn't discover an issue until months after it occurred. I've also had customers ask to reprint a report from their year-end close but that couldn't be done after the year-end close unless, maybe, I could make changes to the routines that would allow it to be done. However, it only took about 5 minutes to run a new backup, restore the year-end backup, run the report, and restore the new backup.

Of course, if you added, deleted, or renamed fields in the middle of the table, you could restore the old backup then restructure the table again - if you like doing the extra work ... and assuming you will be there to do it when the customer needs it.

[sgerber]

Re: Coordinating Development Files with Live Production Files

Just to drag this out a little more...

In my previous post I mentioned the issue of backups. If you are doing "manual" restructuring and there is any possibility that your user might need to restore an old backup for some reason, your best bet is to make minimal changes to existing fields and put new fields at the end of the table. That may not ensure that the old files will still work in the updated app but it will probably work AND it's almost a guarantee that something will NOT work if you insert new fields in the middle of the table.

And be very careful about deciding that an old backup will not be needed. I've had customers who didn't discover an issue until months after it occurred. I've also had customers ask to reprint a report from their year-end close but that couldn't be done after the year-end close unless, maybe, I could make changes to the routines that would allow it to be done. However, it only took about 5 minutes to run a new backup, restore the year-end backup, run the report, and restore the new backup.

Of course, if you added, deleted, or renamed fields in the middle of the table, you could restore the old backup then restructure the table again - if you like doing the extra work ... and assuming you will be there to do it when the customer needs it.

Sounds like good advice, Cal. I wish my customer (my son) took backups at least daily. Have children, Cal? They're the reason the phrase "You can lead a horse to water ..." was coined.

Thanks ... Sam

[CALocklin]

Re: Coordinating Development Files with Live Production Files

Yes, I have two girls. But there still may be hope for your son. My girls were much smarter than me until sometime after they got out of college. Now they are listening again.

(Actually, they are both smarter than me. And it looks like both will have better paying jobs than I ever did - even after accounting for inflation. One graduated from high school as the salutatorian and the other graduated as one of 4 valedictorians - straight "A"s. I didn't even come close. However, even though they may be smarter, they have finally figured out that Mom and Dad, even if just due to experience, do know a bit more about some things than they do. Of course, it may be easier with girls.)

Tell your son I told you to go give him a "Gibbs slap" (I hope you watch NCIS!?) and tell him I said he needs to make backups of his data. Even if he as a RAID drive, he still should make a backup of critical data every night that is either portable (USB drive) and taken off sight or is on-line. If portable, there should be two copies that are rotated; one is taken home every day and the one that was at home is taken back to the office the next day. At the very least, do it once a week. AND - this is one a lot of people ignore - at least one copy a week should be kept for a few weeks and one copy a month should be kept for a year - especially at year-end or if a major change was made to the app. With the low cost of high capacity USB drives today, there's no reason you can't keep probably a year's worth or more of zipped copies of the data on one drive. You don't need to back up the whole folder - just the data files.

[sgerber]

Re: Coordinating Development Files with Live Production Files

Yes, I have two girls. But there still may be hope for your son. My girls were much smarter than me until sometime after they got out of college. Now they are listening again.

(Actually, they are both smarter than me. And it looks like both will have better paying jobs than I ever did - even after accounting for inflation. One graduated from high school as the salutatorian and the other graduated as one of 4 valedictorians - straight "A"s. I didn't even come close. However, even though they may be smarter, they have finally figured out that Mom and Dad, even if just due to experience, do know a bit more about some things than they do. Of course, it may be easier with girls.)

Tell your son I told you to go give him a "Gibbs slap" (I hope you watch NCIS!?) and tell him I said he needs to make backups of his data. Even if he as a RAID drive, he still should make a backup of critical data every night that is either portable (USB drive) and taken off sight or is on-line. If portable, there should be two copies that are rotated; one is taken home every day and the one that was at home is taken back to the office the next day. At the very least, do it once a week. AND - this is one a lot of people ignore - at least one copy a week should be kept for a few weeks and one copy a month should be kept for a year - especially at year-end or if a major change was made to the app. With the low cost of high capacity USB drives today, there's no reason you can't keep probably a year's worth or more of zipped copies of the data on one drive. You don't need to back up the whole folder - just the data files.

Actually, NCIS is my favorite program, so I know all about the "Gibbs Slap". Isn't it amazing how long some series last these days? Maybe not a threat to Gunsmoke but 10-12 years is always a good run.

I'll give my son your message, Cal. Both my kids, however, like to learn the hard way. It usually takes one or two disasters for them to come around.

I'm personally an offender of "Cal's rule" since I work at my home. It's more difficult to take backups off premises. I'm pretty vulnerable to fire or theft leaving me high and dry.

Take care ... Sam

[Ted Giles]

Re: Coordinating Development Files with Live Production Files

I'm personally an offender of "Cal's rule" since I work at my home. It's more difficult to take backups off premises. I'm pretty vulnerable to fire or theft leaving me high and dry.

Asus Storage works well on the Web Sam, and there are other online safe repositories as well.

If you are not easily offended, try viewing Mrs Brown's Boys. on BBC i-player

[CALocklin]

Re: Coordinating Development Files with Live Production Files

I'm personally an offender of "Cal's rule" since I work at my home. It's more difficult to take backups off premises. I'm pretty vulnerable to fire or theft leaving me high and dry.

I used to worry about backups of customer apps more. Then I realized that every customer except "the one I'm working on now" already has the latest and greatest version on their system. So, even if my hard drive fails completely, I can still "recover" the latest and greatest version. I do keep backups here at home but I don't panic anymore over what will happen if even those are lost. My personal stuff is another story - I zip the important stuff periodically or after significant changes/additions and upload it to my website.