Caves Travel Diving Graphics Mizar Texts Cuisine Lemkov Contact Map RSS Polski
Trybiks' Dive Texts DBExpress Checking Dangling Pointers vs. the New Memory Manager YAC Software
  Back

List

Charsets

Charts

DBExpress

Delphi

HTML

Intraweb

MSTest

PHP

Programming

R

Rhino Mocks

Software

Testing

UI Testing

VB.NET

VCL

WPF

Checking Dangling Pointers vs. the New Memory Manager
Some time ago I had an annoying problem with accessing the MySQL database (5.0.32) from Delphi 2007 (all patches applied) - during automated tests we would get intermittent Access Violation errors when assigning a query text to TSQLQuery.SQL.Text. Sometimes this worked without any problems, sometimes DUnit would report an error in one test or another; sometimes the tests worked on one computer, barfed on another... stuff like that. A classic example of an unassigned variable or of referencing freed memory. :-)

The program that used MySQL worked fine, only the tests would fail once in a while. Fortunately, a QualityCentral report had a suggestion on how to fix (or at least ommit) this problem - see QC#58377. It was enough to assign NIL to FMetaData at the end of TDBXConnection.Close in SqlExpr.pas.

This fixed the problem, but for a while I had no time for further investigation. Still, I wanted to know exactly why the fix worked and why without it the application worked as well as it did... So, finally, I got a chance to dig into this deeper.

First thing was to check whether FastMM with the various debug options on finds the problem. Unfortunately, the lastest version (4.90) didn't find anything - with standard options the problem appears and FastMM doesn't report any errors, while in FullDebugMode the problem goes away (which is probably due to different memory handling in that mode).

Perusing the code of SqlExpr.pas it was obvious that FMetaData gets assigned in TSQLConnection.DoConnect, but is never nilled. That isn't neccessarily a problem - FMetaData is only a reference to an object, so if TSQLConnection gets freed before the referenced object does, there shouldn't be any issues. However, FDBXConnection that FMetaData references is freed in TSQLConnection.DoDisconnect. So, if your code references FMetaData after a connection is closed, there will be problems.

But how can FMetaData be referenced after a connection is closed? After all, the database metadata should only be need when a connection is open...

Then I recalled one property in the TSQLConnection - KeepConnection. This parameter, when set to FALSE, causes the driver to drop a connection as soon as an SQL command is finished executing. The driver reconnects automatically when executing consecutive SQL commands, so, theoretically, setting this parameter to FALSE slows down execution a bit, but at the same time keeps the number of simultaneous connections down. (Setting the parameter to TRUE has it's own problems - I'll write about that shortly.)

So, the following code (with KeepConnection set to FALSE)
  LConnection := CreateSQLConnection;
  try
    // First query:
    LQuery := TSQLQuery.Create(NIL);
    try
      LQuery.SQLConnection := LConnection;
      LQuery.SQL.Text := 'select * from TIMING;';
      LQuery.Open;
    finally
      FreeAndNIL(LQuery);
    end;
    // Second query:
    LQuery := TSQLQuery.Create(NIL);
    try
      LQuery.SQLConnection := LConnection;
      LQuery.SQL.Text := 'select * from TIMING;';
      LQuery.Open;
    finally
      FreeAndNIL(LQuery);
    end;
  finally
    FreeAndNIL(LConnection);
  end;
closes the connection after the first query is destroyed, and reopens the connection when the second query is opened. Ah, but that's just the problem - FMetaData is referenced in the second query even before the query itself is opened (when setting the SQL.Text property)...

However, if you would run this code, nothing bad would happen - yes, the object referenced by FMetaData was destroyed, but the memory's intact, so using FMetaData between connections has no ill effects (in code as simple as the one above).

To show the error, we should corrupt memory that FMetaData is pointing to. To do that, we need to know how the memory manager allocates and deallocates memory. Remember then, that the memory manager changed in Delphi recently (thanks to the great job the FastCode people did) - since D2006 a new memory manager - FastMM - is used instead of the old Borland's one.

Under the old memory manager it would've been enough to allocale some random bits of memory and clear those allocations. After a short while, the memory freed by SQL connection's metadata would be reset, thus raising an exception in the second query.

This will not necessarily work with the new memory manager - small memory allocations are divided into pools or buckets. Each allocation is kept in the bucket appropriate for the size of the allocated memory. So, if after freeing a block of memory we were to try to currupt that block by allocating and clearing blocks of a different size, nothing wrong would happen.

Thus, when clearing allocated memory, it would be best to allocate blocks of the same size as TSQLConnection's metadata. Let's change the code above to this:
  // Before the first query:
  LSize := LConnection.MetaData.InstanceSize;
  ...
  // Before the second query:
  SetLength(LMem, CCount);
  for k := 0 to CCount - 1 do
    LMem[k] := NIL;
  try
    for k := 0 to CCount - 1 do
    begin
      GetMem(LMem[k], LSize);
      FillChar(LMem[k]^, LSize, $00);
    end;
  finally
    for k := 0 to CCount - 1 do
      FreeMem(LMem[k]);
  end;
Now, if you run this code, where CCount = 3, you'll get an AV error on the second query. Changing the code of SqlExpr.pas to set FMetaData to NIL fixes things nicely.

To verify the above musings, I ran one more test - will this code throw an exception if we initialize TSQLConnection's KeepConnection to TRUE? Fortunately :-) and as expected, under that setting the code works fine even without the change in SqlExpr.pas.

Top

Comments
Alas!
No comments yet...

Top

Add a comment (fields with an asterisk are required)
Name / nick *
Mail (will remain hidden) *
Your website
Comment (no tags) *
Enter the text displayed below *
 

Top

Tags

DBExpress

Delphi


Related pages

Delphi interfaces... again

Checking "Dangling" Event Handlers in Delphi Forms

Drag-n-drop files onto the application window

Intraweb and MaxConnections

A Case for FreeAndNIL

Intraweb as an Apache DSO module

"Device not supported" in Intraweb

Automated GUI Testing

Rounding and precision on the 8087 FPU

SessionTimeout in Intraweb

Using TChart with Intraweb

Unknown driver: MySQL

TIdMessage's CharSet

Software Guarantees

Automated Testing of Window Forms

TChart - Missing Labels in Axes

Memory Leaks and Connection Explosions in DBExpress

Controlling Conditional Defines and Compilation Switches

Detecting Memory Leaks with DUnit

last_insert_id() and DBExpress

Registering Extensions

DBExpress and Thread Safety

Forms as Frames

Accessing Protected Members

Objects, interfaces, and memory management in Delphi