Massive File Sizes

[davidv43]

I am converting a database from V9 to V10 (which I LOVE!!). However, my file sizes have grown HUGE. The old database with all the data was 29MB. The new database with no data (see attached) is nearly 4GB and takes a long time to compact. The patient master.ddm file alone is nearly 2GB.

This seems like A5 isn't cleaning up after itself. Is anyone else having this problem? I did do a lot of designing and changing of forms, but I deleted those I am not using.

I attached a .RAR file, because 2GB is apparrently too big for .zip.

[csda1]

Re: Massive File Sizes

Hi David,

If you have an excessively large memo file (*.fpt) or large dictionary memo file (*.ddm, *.sem, *.alm), then you have a corrupt memo file associated with the table, set, dictionary or database. Use a backup of that file the associated other 2 files that go with it. Only a used data memo field should ever reach anything close to 2 GB, which is the maximum size for Windows normally.

[davidv43]

Re: Massive File Sizes

I'd use a backup if I had one, but I am actively developing forms and experimenting with new screen layouts (and new desktop gadgets), so my backups are several versions old. I will lose the work I have done on the files.

I guess I don't understand why using the tools to repair the memo file, compact the database, etc don't work on these "corrupt" files.

[csda1]

Re: Massive File Sizes

I'd use a backup if I had one, but I am actively developing forms and experimenting with new screen layouts (and new desktop gadgets), so my backups are several versions old. I will lose the work I have done on the files.

This is why the correct procedure for backups is to keep older ones progressively less, but still there. E.g. one for every year, rotating through 1 for every month of the year (reusing the month each year), 1 for every week of the month, and 1 for every day of the week. Then you can go back to a backup just before the corruption.

I guess I don't understand why using the tools to repair the memo file, compact the database, etc don't work on these "corrupt" files.

If it's a table fpt file, you just need a new data file with the fpt file, the dictionary files should be alright for that table and can be copied with all the forms and operations there. The reverse is also true if the dictionary is there. (Exception might be Index names longer than 10 characters may e truncated)

If you really need recovery services for the global code and things stored in the AL* files, I'm probably the only one with tools that can potentially recover that from the *.alm file.

One thing you can try is create a new table or set, then copy the forms and other objects from one to the other.

[davidv43]

Re: Massive File Sizes

Ira,

thank you for the help. I don't guess that I am explaining myself well. I am currently writing a database and designing forms for use with it. I may redesign a form 10 times before it looks and works the way I want it to.

The file which is too big is the DDM file. I don't know what's in it, but when I replaced it with an older version, the forms don't work at all correctly.

[csda1]

Re: Massive File Sizes

Hi David,

The DDM, DDX, and DDD files all go together. You can't change one without the other. They all have the table's layouts, operations, fieldrules etc., but no data.

You may be able to use the duplicate table structure to copy the good stuff, but it may not work.

If not, you may be able to copy the layouts and operations individually to another table to recover them, then use the recovered ones in a recreated set of dictionary files.

Compacting a corrupted memo file (DDM in this case) may not help at all.

Recovery is an expensive process, either time or money. How much time does a backup take? Why do you think my CSDA Code Utility has a backup of Data Dictionaries feature?

[davidv43]

Re: Massive File Sizes

Ira,

I sincerely appreciate all your help through the years. You have always been a reliable and valuable voice on these boards.

My question is this: when can I expect file corruption to happen? If it happens only during development when one is changing forms, field rules, etc, then fine. I can handle that - although constantly monitoring your file size to make sure it hasn't become corrupted adds to development time substantially and negates much of the purported benefit of developing with A5.

If corruption happens during routine use of the database after development is finished, that's a real problem. I have never had to restore an Access database (of which I have several) unless there was a hardware failure. No mistake, A5 beats Access hands-down in terms of capability, but I don't have time to constantly monitor the files for corruption. I need a fairly maintanence-free system once the development is complete.

With all respect, if you can offer a product which backs up and restore critical files, why can't Alpha?

It is possible that I'm the only one who has experienced this type of corrupt files and file size bloat, and if that is the case, then I will accept my bad luck and re-do the work I've done with confidence that it won't happen again.

Dave

[davidv43]

Re: Massive File Sizes

One more question - does eliminating memo fields in the database solve anything?

[Tom Cone Jr]

Re: Massive File Sizes

David, memo field bloat has been discussed a number of times in the various desktop forums. Finding the threads is difficult because of the different ways folks describe the issue. Dr. Wayne has the definitive article on bloat prevention at www.learn alpha.com. Look for an article there called "memos and browses that work". The procedure outlined there should be used to protect user entered memo data.

In your case the corruption seems to have occurred during development of the database. The article just mentioned won't help you much with that. The best course during development is frequent backups throughout each development session. If you write scripts and functions like me, they fail many times before I "get them right". These failures can leave Alpha Five in an unstable condition and lead to problems. Having a recent backup handy is a godsend. And, it's easy to tell from the size of the resulting zip file whether the backup itself contains a bloated memo file. Ira's utility makes it dead easy to make interim backups (dictionaries only, or dictionaries + data).

[davidv43]

Re: Massive File Sizes

Thanks, Tom. To the question of how stable the system is once development is complete, I assume you and Ira find it stable enough to warrant heavy use.

[csda1]

Re: Massive File Sizes

Hi David,

To the question of how stable the system is once development is complete, I assume you and Ira find it stable enough to warrant heavy use.

I have never seen the data dictionary memo field (*.alm, *.sem, *.ddm) ever grow beyond say 30 Megabytes. Compacting reduces it down to smallest possible sizes for what is stored. I'd say it's extremely stable. And believe me, I am one of the few (Cal Locklin, and Bill Parker being others), that fool with the data dictionaries directly, and have never had any issues.

[CALocklin]

Re: Massive File Sizes

I've never had problems with a data dictionary memo file either so I'll agree that it's rare.

I have seen problems with a memo field for the data but even that was a long time ago. Even the one large "problem" v9 application (strange errors and crashes that seem to be memory related) of mine that is used by multiple small companies has never had a memo file corruption even though there are memos related to the two main tables and multiple memos are updated daily. (Uh oh, I hope I didn't jinx myself by saying that!:()

Due to those past problems I still try to keep memo field usage to a minimum, never use RTF memos (just because I find them quirky), and for all but the simplest tables, keep the memos in a separate table a la "memo fields that work" just so I don't have to worry about losing the rest of the data if the memo file gets corrupted. (Although I'm pretty sure now that that, too, could be overcome.)

Something that may or may not be helping me is that my newer apps "override" the default memo editing. In some of them I don't allow editing in the basic memo window but, instead, require the user to click the "pencil icon" which I've replaced with a "user control" that opens a separate xdialog window for editing the memo. The original reason for doing this was to see if it improved the reliability of memos because it was using table mode access instead of form mode (I can't tell if it did or not) but I then discovered that I could have a memo editor that would allow the user to either view or edit the memo without putting the main form into Change mode. That had been a pet peeve of mine for a long time - just opening the full memo for viewing would put the form into Change mode then users would get confused wondering what they had changed - so now I use this on most new apps that use plain text memos. The routine also allows the user to insert a new line with date or date/time at either the top or bottom of the memo depending on which argument the developer (me) sets.