file locking

[totalbiz]

I searched the archive here for this error message which i am getting:

File locking record count exceeds Max 512

and it seems to have no solution other than turning off referential integrity which i did. It fixed it. I have some records with a couple of thousand child records attached. Is that the only solution or did i miss a post with the fix.

[Davidk]

I don't have the experience with A5 to answer this but... the error message refers to file locking... not record.

I would hope A5 won't grab a record and try to lock thousands of child records... it just doesn't work that way... I hope.

I would expect (I suppose) a limit to how many files get locked in a session and it seems that's what you're hitting. Now... why? What are you doing that would open that many files... or maybe the same file in a loop?

[Al Buchholz]

Yes that is one of the issues with referential integrity and apparently you have a parent with more than 512 children.....

You'll find on the messageboard many threads about work arounds. Turn off ref int and build your own delete the children routine.

[Davidk]

Yikes... looks like this issue has been around for years... any reason ever given for the way it is?

[Al Buchholz]

It's a fundamental dbase issue with referential integrity. Who would ever have a parent record with more than 250 children? Not back when this was designed.....

[Occam49]

Hmmm, how about a check register with master record for the account, and a detail record for each check. This could easily run into multiple thousands of detail records.

[csda1]

Hi Darrell,

Hmmm, how about a check register with master record for the account, and a detail record for each check. This could easily run into multiple thousands of detail records.

I believe the limitation may actually be Windows and/or the Windows file/record locking mechanism.

You don't really need referential integrity for the case you state above. Referential Integrity basically means that when you lock a parent, it's connected children that have referential integrity on also are locked, keeping others from modifying the children. Also, deletes of the parent cascade down to the children as well.

For many-to-many relationships (e.g invoice header to invoice line items to inventory), if you locked the inventory record (by referential integrity), no one else could use that part in another invoice until you were done with it, a bad thing.

[Occam49]

Referential Integrity basically means that when you lock a parent , it's connected children that have referential integrity on also are locked, keeping others from modifying the children. Also, deletes of the parent cascade down to the children as well.

I respectfully disagree. Referential integrity (RI) says nothing about locking or any other implementation specific details.

Simply RI states that: 1) A parent record with a primary key must exist to create child record where a foreign key is defined in the child referencing the parent primary key. 2)Deleting the parent record implies deleting the children, and 3.) If the primary key in the parent should change, the foreign keys in the related child records must also change.

Further more, the only data elements of child or parent records under RI constraint are the primary key in the parent, and the foreign keys in the child, all other data elements are free from RI constraint and may be modified. This does not imply that there may be some other reason to lock records, but that is not a RI requirement. In general, file system record locking is a primitive brute force mechanism that results in very poor concurrency performance.

In my check register example, without RI constraints being enforced by some mechanism, the master record could be deleted leaving orphaned child records.

[SOAP BOX ON]

I'm new to A5, and not sure how the A5 native file system is implemented, but it seems to be based upon primitive system level file record [I.e. byte range] locking which will never be able to provide efficient or comprehensive RI enforcement in a multi-user application. The fundamental flaw of file system oriented database systems is that all active user applications have write access to the data on disk. The OS knows nothing about the data semantics, it can only read, write and lock data at the byte level on disk

A reliable multi-user database system requires a database engine that is separate from the application that is controlling and managing access of all data and enforcing the declarative RI [and other] rules and insuring transaction integrity using an intelligent write-through global record cache and a transaction log. In this scenario, only the engine [normally] ever writes, updates, or deletes data. This is how ALL DBMS systems work. The global LRU cache minimizes reads as multiple users applications may find the data in the cache avoiding an I/O. Frequently accessed data stays in the cache, unused cached data is over written by new requests for data. The transaction log only writes actual changed data to disk in sequential file which is very fast. In a transaction oriented dbms engine, transaction [covering potentially many separate data updates to many records and indexes,etc] either completes as a whole or fails as a whole, greatly reducing the chances of data corruption. The dirty pages in the global data cache is written to disk at check point intervals reflecting many updates to a given page improving I/O effeciency. The previous description is very much simplified, but is part of the essence of engine oriented dbms.

Using native A5 tables in a busy multi-user environment will ultimately result in data corruption [due to failed multi-table or index writes, or deletes] and poor concurrency performance[due to data locking].

Even in a lightly used system, the data integrity protection gained by using a database engine is well worth the additional complexity. Ever try rolling back even a days worth of erroneous updates with a file oriented database? Or, how about recovering to the exact point in time of the last committed transaction after a disk crash when you last backup was 24 or more hours ago without losing or re-entering a single committed transaction? I've done both simply and reliably when using Oracle and SQL Server ... now looking at PostgreSQL for use with A5
[SOAP BOX OFF]

[csda1]

I respectfully disagree. Referential integrity (RI) says nothing about locking or any other implementation specific details.

That is correct. I'll restate it to say that Alpha's implementation does.

Simply RI states that: 1) A parent record with a primary key must exist to create child record where a foreign key is defined in the child referencing the parent primary key. 2)Deleting the parent record implies deleting the children, and 3.) If the primary key in the parent should change, the foreign keys in the related child records must also change.

That is correct too. But we are talking about a file sharing system called Windows, not an SQL or similar (although Alpha can now utilize that) using a very primitive structure of dBase II/III/IV.

Further more, the only data elements of child or parent records under RI constraint are the primary key in the parent, and the foreign keys in the child, all other data elements are free from RI constraint and may be modified.

That too is correct, but not practical in an OS that basically allows record locking, but not field locking. It could implement a field lock, but you would see slowdowns probably in an order of magnitude or greater.

This does not imply that there may be some other reason to lock records, but that is not a RI requirement. In general, record locking is a primitive brute force mechanism that results in very poor concurrency performance.

Primitive, yes. Poor concurrency performance, I'd say not. Many people have used Alpha Five with 100 users banging on it constantly using the native DBF and received acceptable performance (although bad coding can bring it to a crawl, as it could any system)

In my check register example, without RI constraints being enforced by some mechanism, the master record could be deleted leaving orphaned child records.

If there was a user accessible lock/unlock function in A5 (which unfortunately there is not), then code could do this, but the typical users of Alpha 5 can't typically do this even if it existed. Easiest way to do this in A5 is to open a set with referential integrity (Alpha's) on, and delete the children

[SOAP BOX ON]

I'm new to A5, and not sure how the A5 native file system is implemented, but it seems to be based upon primitive system level file record [I.e. byte range] locking which will never be able to provide efficient or comprehensive RI enforcement in a multi-user application. The fundamental flaw of file system oriented database systems is that all active user applications have write access to the data on disk. The OS knows nothing about the data semantics, it can only read, write and lock data at the byte level on disk

A reliable multi-user database system requires a database engine that is separate from the application that is controlling and managing access of all data and enforcing the declarative RI [and other] rules and insuring transaction integrity using an intelligent write-through global record cache and a transaction log. In this scenario, only the engine [normally] ever writes, updates, or deletes data. This is how ALL DBMS systems work. The global LRU cache minimizes reads as multiple users applications may find the data in the cache avoiding an I/O. Frequently accessed data stays in the cache, unused cached data is over written by new requests for data. The transaction log only writes actual changed data to disk in sequential file which is very fast. In a transaction oriented dbms engine, transaction [covering potentially many separate data updates to many records and indexes,etc] either completes as a whole or fails as a whole, greatly reducing the chances of data corruption. The dirty pages in the global data cache is written to disk at check point intervals reflecting many updates to a given page improving I/O effeciency. The previous description is very much simplified, but is part of the essence of engine oriented dbms.

Using native A5 tables in a busy multi-user environment will ultimately result in data corruption [due to failed multi-table or index writes, or deletes] and poor concurrency performance[due to data locking].

Even in a lightly used system, the data integrity protection gained by using a database engine is well worth the additional complexity. Even try rolling back even a days worth of erroneous updates with a file oriented database? Or, how about recovering to the exact point in time of the last committed transaction after a disk crash when you last backup was 24 or more hours ago without losing or re-entering a single committed transaction? I've done both.
[SOAP BOX OFF]

Use SQL. It's more than the typical SOHO market needs, but the great thing about A5v9 is that you can do it all.

[Davidk]

Rolling your own RI is not the worse thing in the world but there should be a warning somewhere that you're out of luck when child record counts grow beyond the maximum if you're using A5 RI. I've not seen that maximum in any literature... including the new grid just released. That's not a lot of records and having 512 child records is not a reason to move to SQL.

[Larry Gordon]

Hi,

Does the above thread still apply today? I am now getting this error message when attempting to amend one specific record. My set consists of a parent table with about six child tables and two grand-child tables. However, none of the child tables have 512 records, though adding the total records from several of the child tables will reach the 512.

In any event, is my only remedy to get rid of the RI?

Also, is having a few orphaned records a big deal?

Many thanks,

Larry

[Larry Gordon]

Hi,

Does the above thread still apply today? I am now getting this error message when attempting to amend one specific record. My set consists of a parent table with about six child tables and two grand-child tables. However, none of the child tables have 512 records, though adding the total records from several of the child tables will reach the 512.

In any event, is my only remedy to get rid of the RI?

Also, is having a few orphaned records a big deal?

Many thanks,

Larry

[DaveM]

Have hit this yeards ago and ref int was turned off in affected sets and finally all sets. Built a cleanup script to delete the child records when needed. was run on the delete button in possibly affected forms. I still do that today.