tag:blogger.com,1999:blog-67890562814400072482024-03-07T18:45:26.129+00:00EurekaLog's blogYour place for information about EurekaLog and debugging.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comBlogger114125tag:blogger.com,1999:blog-6789056281440007248.post-69620692679979500722024-03-07T18:44:00.004+00:002024-03-07T18:44:35.440+00:00The Secret of Access DeniedA client contacted us and said that he could not install EurekaLog. More precisely: EurekaLog is installed, but subsequent launch of the IDE raises an error:
<blockquote>Can't load package C:\Program Files (x86)\Neos Eureka S.r.l\EurekaLog 7\Packages\Studio25\EurekaLogExpert250.bpl.
Access is denied.</blockquote>
Moreover, the client claimed that the file exists, there is access to it, reinstalling EurekaLog does not help.<br />
<a name='more'></a><br />
Additionally, it was found that the code from the <code>EurekaLogExpert250.bpl</code> package actually does not receive control (is not executed).<br />
<br />
The IDE uses the same <code>LoadPackage</code> function from the <code>SysUtils</code> unit that you (your code) use. The IDE does not execute any secret code. The <code>LoadPackage</code> function is implemented as follows:
<pre class="brush:delphi">function LoadPackage(const Name: string; AValidatePackage: TValidatePackageProc): HMODULE;
begin
Result := SafeLoadLibrary(Name);
if Result = 0 then
raise EPackageError.CreateResFmt(@sErrorLoadingPackage, [Name, SysErrorMessage(GetLastError)]); // -----
try
InitializePackage(Result, AValidatePackage);
except
FreeLibrary(Result);
raise;
end;
end;</pre>
where the <code>InitializePackage</code> function is implemented as:
<pre class="brush:delphi">procedure InitializePackage(Module: HMODULE; AValidatePackage: TValidatePackageProc);
type
TPackageLoad = procedure;
var
PackageLoad: TPackageLoad;
begin
CheckForDuplicateUnits(Module, AValidatePackage);
@PackageLoad := GetProcAddress(Module, 'Initialize'); //Do not localize
if Assigned(PackageLoad) then
PackageLoad
else
raise EPackageError.CreateFmt(sInvalidPackageFile, [GetModuleName(Module)]);
end;</pre>
As you can see: the only place where an exception with an error code from the OS (5 = <code>ERROR_ACCESS_DENIED</code>) can be raised when loading a package is the marked line. This means that an Access Denied error when loading a package can only occur if the call to the <code>LoadLibrary</code> function fails.<br />
<br />
Therefore, the client was asked to create a new empty VCL application with the following test code:
<pre class="brush:delphi">procedure TForm1.Button1Click(Sender: TObject);
var
Lib: HMODULE;
begin
Lib := LoadPackage('C:\Program Files (x86)\Neos Eureka S.r.l\EurekaLog 7\Packages\Studio25\EurekaLogExpert250.bpl');
if Lib = 0 then
RaiseLastOSError
else
MessageBox(0, 'The package was loaded!', 'Test', 0);
end;</pre>
As it turns out, the test application loads the package successfully.<br />
<br />
To be sure that the IDE was not executing any additional hidden code, we asked the client to create a new Design-Time package with a new unit containing only the test code:<br />
<pre class="brush:delphi">unit Unit1;
procedure Register;
implementation
uses
Windows, SysUtils;
procedure Register;
var
Lib: HMODULE;
begin
Lib := LoadPackage('C:\Program Files (x86)\Neos Eureka S.r.l\EurekaLog 7\Packages\Studio25\EurekaLogExpert250.bpl');
if Lib = 0 then
RaiseLastOSError
else
MessageBox(0, 'The package was loaded!', 'Test', 0);
end;
end.</pre>
And when the client tried to load this test package, the IDE again gave an error message.<br />
<br />
Let's make an intermediate result:
<ul><li>The test application successfully loads the package;</li><li>The test package (and IDE) cannot load the package.</li></ul>
This strongly suggests that the problem is not with the package, but with the IDE itself.<br />
<br />
<blockquote>We also tried the following:
<ol>
<li>Launch the IDE (without EurekaLog);</li>
<li>Open the Run / Load Process menu item;</li>
<li>Specify <code>bds.exe</code> as the target process for debugging;</li>
<li>Launch the second instance of the IDE. In this case: the first instance will be the debugger, and the second will be the debugged one;</li>
<li>In the process being debugged: try to add/load the <code>EurekaLogExpert250.bpl</code> package;</li>
<li>In the debugger: monitor for exceptions that occur.</li>
</ol>
This algorithm did not provide any new information. In the debugger: we saw that in the process being debugged, the <code>LoadLibrary</code> function returns 0, which leads to an error being raised - as we assumed above.</blockquote>
Since both we and the client had already checked the access rights to the file <code>EurekaLogExpert250.bpl</code> a hundred times and saw no problems, we had to do something else.<br />
<br />
The <code>EurekaLog<b>Expert</b>250.bpl</code> package is a Design-Time package. It depends on the Run-Time package <code>EurekaLog<b>Core</b>250.bpl</code>. This means that if the <code>LoadLibrary</code> function loads the <code>EurekaLogExpert250.bpl</code> package, it will see that the <code>EurekaLogExpert250.bpl</code> package references the <code>EurekaLogCore250.bpl</code> file and will try to load it too. And if a problem arose while loading the <code>EurekaLog<b>Core</b>250.bpl</code> package, it will bubble up to the top level and will be returned to the caller by the <code>LoadLibrary</code> function.<br />
<br />
Therefore, the client also checked the file <code>EurekaLogCore250.bpl</code> (located in the <code>C:\Windows\System32</code> folder) and found no problems.<br />
<br />
Next, we asked the client to use Microsoft/SysInternals Process Monitor tool to monitor how the IDE was accessing files. It was necessary to launch the IDE, open the adding component dialog, then launch the Process Monitor tool and specify the "Process Name=bds.exe => include" filter. Then try to add the package, see the error message and save the resulting report.<br />
<br />
In the resulting report, we ran a search for the "Eureka" word and immediately saw that the <code>bds.exe</code> was able to open and read the <code>EurekaLogExpert250.bpl</code> file - which further confirms that there is nothing wrong with the <code>EurekaLogExpert250.bpl</code> file.<br />
<br />
But the lines immediately below confused us: Process Monitor reported that the <code>bds.exe</code> is trying to access the <code>EurekaLogCore250.bpl</code> from IDE's <code>\bin</code> folder. This is strange because modern versions of EurekaLog do not touch IDE's <code>\bin</code> folder.<br />
<blockquote>(Very) old versions of EurekaLog installed the <code>ecc32.exe</code> file in IDE's <code>\bin</code> folder. We had to remove this behavior because <a title="Starting Delphi Or C++ Builder Results In Product Or License Validation Error" href="https://blogs.embarcadero.com/starting-delphi-or-c-builder-results-in-product-or-license-validation-error/">Modern versions of the IDE may throw an integrity/license verification error if there are third-party files in IDE's <code>\bin</code> folder</a>:<br / >
<blockquote>Question: I’ve installed and registered C++ Builder or Delphi, yet when it starts I am brought to a web page with the error “Product or License Validation Error”. How can I fix this?<br />
Answer: By far the most common cause for this error is having files or applications not provided by Embarcadero that are copied into the bin folder below where RAD Studio is installed. Only files provided by Embarcadero may reside in the bin folder.</blockquote>
Therefore, modern versions of EurekaLog do not copy any files to IDE's <code>\bin</code> folder.</blockquote>
But when we tried to check the access rights to the <code>\bin\EurekaLogCore250.bpl</code> file, we... did not find such file!<br />
<br />
As it turns out: the <code>\bin\EurekaLogCore250.bpl</code> is, in fact, a directory! The mystery is solved!<br />
<br />
So, the following happened:<br />
<ol>
<li>The IDE calls the <code>LoadPackage</code> function to load the package;</li>
<li>The <code>LoadPackage</code> function calls the <code>LoadLibrary</code> function to load the <code>EurekaLogExpert250.bpl</code> file;</li>
<li>The <code>LoadLibrary</code> function (successfully) opens and reads the <code>EurekaLogExpert250.bpl</code> file;</li>
<li>The <code>LoadLibrary</code> function sees that the <code>EurekaLog<b>Expert</b>250.bpl</code> file contains a link/reference to the <code>EurekaLog<b>Core</b>250.bpl</code> file;</li>
<li>The <code>LoadLibrary</code> function tries to load the <code>EurekaLogCore250.bpl</code> file;</li>
<li>The <code>LoadLibrary</code> function uses <a href="https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order" title="Dynamic-link library search order">standard operating system's library search rules</a> and sees that <code>EurekaLogCore250.bpl</code> is located right here: in IDE's <code>\bin</code> folder;</li>
<li>The <code>LoadLibrary</code> function tries to load <code>EurekaLogCore250.bpl</code> from IDE's <code>\bin</code> folder;</li>
<li>The <code>LoadLibrary</code> function returns error 5 (Access Denied) because the <code>EurekaLogCore250.bpl</code> <b>folder</b> in IDE's <code>\bin</code> folder does not have the "EXECUTE" access right".</li>
</ol>
<br />
And, indeed, everything worked after deleting the <code>EurekaLogCore250.bpl</code> folder from IDE's <code>\bin</code> folder.<br />
<br />
But where did the <code>EurekaLogCore250.bpl</code> folder in IDE's <code>\bin</code> folder came from? I don't know. The EurekaLog installer is made using InnoSetup. Of course, the InnoSetup installation file for EurekaLog has no idea about IDEs you have installed and folders in which they are installed. And therefore the InnoSetup installation file for EurekaLog will not be able to create any folder there - simply because it does not know where it is.<br />
<br />
Registration of EurekaLog in the IDE is carried out by the standalone registration program: <code>.exe</code> file compiled in Delphi from our source code. The problem is that in our source code for EurekaLog registration does not have a single call to <code>MkDir</code> or <code>ForceDirectories</code> functions: the EurekaLog registration application does not create directories, it copies files and enters data about them into Windows registry.<br />
<br />
Even if the <code>EurekaLogCore250.bpl</code> in IDE's <code>\bin</code> folder was created by our installer: why is it a directory and not a file? And why is it alone? Why are there no <code>EurekaLogExpert250.bpl</code> and <code>EurekaLogComponent250.bpl</code> - after all, these three files go together.<br />
<br />
So far it looks like <i>someone</i> has created a <code>EurekaLogCore250.bpl</code> directory in IDE's <code>\bin</code> folder. Perhaps it was some kind of IDE expert?<br />
<br />
If you are our client, if you saw this error and know who creates the <code>EurekaLogCore250.bpl</code> directory in IDE's <code>\bin</code> folder - please <a href="https://www.eurekalog.com/support.php" title="Contact Us">let us know</a>.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-1288082466453212372023-12-07T17:14:00.002+00:002023-12-07T17:14:41.463+00:00Beware of secondary exceptionsWe were contacted by a client who complained that EurekaLog was generating an error report in the wrong place. In fact, the client had an <b>expected</b> exception that he wanted to hide by showing a simple message instead. The client kindly showed his code:<br />
<pre class="brush:delphi">try
Query.Delete; // - an exception is raised here
except
Query.Transaction.Rollback;
ShowMessage('Sorry, could not delete the report');
Exit;
end;</pre>
What's happening? Does EurekaLog really ignore user code?<br />
<a name='more'></a><br />
If the client code were written like this:<br />
<pre class="brush:delphi">try
Query.Delete; // - an exception is raised here
except
Exit;
end;</pre>
then there would be no problem: <code>Query.Delete</code> raises an exception, the exception is caught by the <code>except</code> block, which does nothing, and execution continues further, EurekaLog does not react (*). <br />
<br />
However, the client code looks different: its <code>except</code> block does some (complex) exception recovery.<br />
<br />
In any case, such exceptions are called handled because they are handled by the user's (client's) code. A handled exception is handled by cleanup code, which performs some recovery/rollback actions, after which program execution continues and the exception itself is removed.<br />
<br />
In general, any cleanup code (be it a destructor or a rollback/recovery code) should be written in a such way so it will not raise exceptions. It seems to me that the logic here is obvious: if your <i>normal</i> code raises an exception, then you can always suggest a possible recovery actions (the so-called “Plan B”). For example, if an exception occurs when opening a document - then the document needs to be closed; If an exception occurs while accessing the printer - the print job must be canceled. If there is an error opening a file - you can try to open it again after a pause or user's action. And so on.<br />
<br />
But what will you do if an error occurs in the “Plan B” itself? You wanted to cancel the print job, but it... is not cancelled. So what now? You will not be able to offer any reasonable recovery actions. Because you don't know what happened. Your variables may be corrupted. Hell, you might not even be able to allocate a block of memory right now! The only possible clean way is to kill/restart the application.<br />
<br />
This is why the cleanup code must NOT raise exceptions. Because you can't do anything meaningful with these exceptions. In other words: the cleanup code must handle all exceptions that it knows how to handle.<br />
<br />
But in practice, things may not be the same as in an ideal world. <br />
<br />
In this case, the client's code raises a <b>second</b> exception inside <code>Query.Transaction.Rollback</code>. It means that the execution of the <code>except</code> block is interrupted and execution moves to the exception handler higher up the stack. It turns out that there are no more <code>try</code> blocks higher up the stack in the client's code, so execution is transferred to the global exception handler, which calls EurekaLog to generate a bug report.<br />
<br />
Correcting the code will involve following the rule “cleanup code must not raise exceptions” (“cleanup code must handle known exceptions”). For example, as follows:<br />
<pre class="brush:delphi">try
Query.Delete; // - an exception is raised here
except
try
Query.Transaction.Rollback;
except
on EDatabaseError do; // - ignore database errors
end;
ShowMessage('Sorry, could not delete the report');
Exit;
end;</pre>
In this case, the second exception will be handled in place and will not "bubble up" to the next exception handler (be it a <code>try</code> block or a global exception handler). It is important to write the correct code - i.e. so that it doesn't block exceptions that you don't know how to handle. For example, the following code is incorrect:<br />
<pre class="brush:delphi">
try
Query.Transaction.Rollback;
except
// - ignore ALL errors
end;</pre>
It is incorrect because you do not know how to properly handle/rollback/recover from, for example, an <code>EAccessViolation</code> exception. It is best to write the code so that it takes into account the most narrow conditions for exceptions - only those that you know exactly how to handle. For example, if you know a specific type/code of the error when canceling a transaction - then write like this:<br />
<pre class="brush:delphi">
try
Query.Transaction.Rollback;
except
on E: ESomeSpecificDatabaseException do
if E.ErrorCode = 1234 then
begin
// - ignore only this specific error
end
else
raise;
end;</pre>
<br />
<br />
<br />
Remarks:<br />
<ul>
<li>(*) ...unless the <a href="https://www.eurekalog.com/help/eurekalog/advanced_page.php" title="EurekaLog Advanced Options">Catch Handled Exceptions option is not enabled in EurekaLog</a>. When the Catch Handled Exceptions option is enabled - EurekaLog will show bug reports about all exceptions, even if they were handled by <code>except</code> blocks. This option is intended for local debugging, as an aid in finding a "bad" code. It should not be used in production.</li>
<li>If you don't need to do special cleanup after an exception, but need to block an expected exception, then you can <a href="https://www.eurekalog.com/help/eurekalog/how_to_ignore_particular_exception.php" title="How to ignore a particular exception">ignore it</a>, or <a href="https://www.eurekalog.com/help/eurekalog/how_to_mark_exception_as_expected.php" title="How to mark exceptions as expected">mark it as "expected"</a>.</li>
<li>If you do not fix your code so that your recovery/handling code does not raise exceptions, then you will have secondary exceptions. EurekaLog has <a href="https://www.eurekalog.com/help/eurekalog/nested_exceptions_page.php" title="Exceptions Options">a set of options that control EurekaLog's behavior for such exceptions</a>.</li >
</ul>GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-44132754038275330032023-11-17T08:59:00.001+00:002023-11-17T08:59:43.298+00:00Black Friday and Cyber Monday SaleWe are pleased to announce Black Friday and Cyber Monday <b>30% discount</b> on any of our EurekaLog products using the coupon code found below.<br />
<br />
The sale starts on Black Friday (November, 24) and ends at the end of Cyber Monday (November, 27).<br />
<br />
Enter this code when paying for the item on our web site:<br />
<br />
<b>BFCM2023</b><br />
<br />
<hr />
<br />
Existing customers with valid or expired licenses can log in and purchase upgrades, new licenses and extensions here:<br />
<br />
<a href="https://www.eurekalog.com/login.php" title="Log in to customer control panel">https://www.eurekalog.com/login.php</a><br />
<br />
Use the login credentials we sent you at purchase time.<br />
<br />
<hr />
<br />
New customers (without existing licenses) can use the discount code here:<br />
<br />
<a href="https://www.eurekalog.com/buy.php" title="Purchase EurekaLog">https://www.eurekalog.com/buy.php</a>GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-76139773025790387762023-11-10T13:39:00.002+00:002023-11-10T13:39:35.432+00:00EurekaLog 7.12 is outWe are pleased to announce the availability of the new 7.12 version.<br />
<a name='more'></a><br />
<div style="text-align: center;">
<a href="https://www.eurekalog.com/" target="_blank" title="EurekaLog Logo"><img alt="EurekaLog Logo" src="https://www.eurekalog.com/images/logo.png" height="68" width="295" /></a></div>
<blockquote>
EurekaLog is a tool for Delphi and C++Builder that gives your application the power to catch every exception and memory/resource leak, generating a detailed log of the call stack (with unit, procedure and line number), optionally sending you a copy of each log entry via email or the Web via the most used Web Bug-Tracking tools (as Mantis, BugZilla, FogBugz, JIRA, YouTrack, Redmine, Exceptionless, GitLab, and GitHub).</blockquote>
EurekaLog represents the most advanced exception and memory leaks logger technology available for Delphi/C++Builder developers.<br />
<br />
To learn more about EurekaLog, please visit our website at:<br />
<a href="https://www.eurekalog.com/" target="_blank" title="https://www.eurekalog.com">https://www.eurekalog.com</a><br />
<br />
To download a demo, please visit:<br />
<a href="https://www.eurekalog.com/downloads.php" target="_blank" title="https://www.eurekalog.com/downloads.php">https://www.eurekalog.com/downloads.php</a><br />
<br />
If you would like to purchase a new license, please visit:<br />
<a href="https://www.eurekalog.com/buy.php" target="_blank" title="https://www.eurekalog.com/buy.php">https://www.eurekalog.com/buy.php</a><br />
<br />
<strong>Changes in 7.12 build:</strong><br />
<ol>
<li>Added: <b>[IMPORTANT]</b> Added workflow path from CLOSED to NEW/OPEN/REOPENED for some bug trackers. <a href="https://www.eurekalog.com/help/eurekalog/bug_tracker_workflow.php" title="Bug Tracker Setup / Issue workflow">Please see</a></li>
<li>Added: <b>[IMPORTANT]</b> Added ability to catch non-exception crashes in C++ Builder, such as Abnormal Program Termination. <a href="https://www.eurekalog.com/help/eurekalog/configuring_cppbuilder.php" title="Configuring C++ Builder project for EurekaLog">Please see</a></li>
<li>Added: Support for RAD Studio 12</li>
<li>Added: Recompiled for RAD Studio 11 update 3</li>
<li>Added: New option for EurekaLog's behaviour when processing secondary exceptions during exception's processing</li>
<li>Added: Completely reworked internals of calling default RTL handler to allow proper feature functioning in all cases</li>
<li>Added: EurekaLog will now show call stack dump when EIP/RIP is invalid</li>
<li>Added: New variable: bug location (address); variable name: _Location</li>
<li>Added: "Reserve lower memory" option</li>
<li>Added: Support for mORMot2 debug info format</li>
<li>Added: Now user's address will be used as "Reply-to" header when sending bug reports</li>
<li>Added: --el_wait command line switch with default value of 60 seconds. Now EurekaLog will wait for executable file to become unlocked/writable (to workaround anti-viruses locking the compiled file)</li>
<li>Added: AddCustomDataToBugReport and AddCustomWebField functions (EEvents unit)</li>
<li>Added: "Largest Available Memory Block" value to bug reports</li>
<li>Fixed: BugZilla API was upgraded to a modern JSON API</li>
<li>Fixed: Possible rare range-check error</li>
<li>Fixed: Sending to Exceptionless fails with EVariantOverflowError on old IDEs</li>
<li>Fixed: Possible memory leaks when SMTP send fails</li>
<li>Fixed: Very rare memory errors for TEurekaExceptionInfo</li>
<li>Fixed: Various improvements for handling stack overflows, synchronize exceptions, multi-threaded call stacks</li>
<li>Fixed: Possible access violation when taking call stack of already shutdown thread</li>
<li>Fixed: Various multithreading issues</li>
<li>Fixed: [SMTP] name in the HELO command is now a fully qualified domain name</li>
<li>Fixed: Possible EurekaLog dialog hang when main window is not responding</li>
<li>Fixed: [x64] Debug info providers may fail to obtain information from modules loaded at high addresses (above 4 Gb)</li>
<li>Fixed: Rare hang in MS debug info provider startup</li>
<li>Fixed: Very rare memory leak</li>
<li>Fixed: Very rare range-check error</li>
<li>Fixed: EurekaLog may fail to read map file property (very rare)</li>
<li>Fixed: Empty network information in leak reports</li>
<li>Fixed: Old IDEs fail to decrypt passwords from options during leak reporting, leading to various issues</li>
<li>Fixed: Dialog, Log Builder and Send Engines classes will now expand exception-specific variables</li>
<li>Fixed: EurekaLog dialog may change DPI awareness of the thread</li>
<li>Fixed: [C++ Builder] TLS detection code could produce incorrect results in some cases; this leads to EurekaLog being disabled</li>
<li>Fixed: [Viewer] Possible Access Violation when there are unsaved changes</li>
<li>Fixed: [Viewer] Very rare range check error on opening some reports</li>
<li>Fixed: [Regression] Possible memory leak when both memory leaks and resource leaks are enabled</li>
<li>Fixed: [Obsolete] Rare Access Violation when using EAppMultiThreaded (e.g. "Auto-handle TThread exception" option is enabled)</li>
<li>Fixed: Various minor and internal improvements</li>
<li>Changed: Signature for OnPasswordRequest event</li>
</ol>GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-22135216361297592192023-09-06T22:26:00.006+01:002023-09-07T11:16:23.255+01:00Creating an API (contract) for your own DLL<h3>Or: don't create your own DLLs without reading this article!</h3>
<br />
<b>This article is not about EurekaLog, but about writing your own DLLs in general</b>. This article is based on questions on the forums: "How do I return a string from a DLL?", "How do I pass and return an array of records?", "How do I pass a form to a DLL?".<br />
<br />
So that you do not spend half your life figuring it out - in this article I will bring everything on a silver platter.<br />
<br />
<b><font color="red">Important note</font>:</b> the article must be read <b>sequentially</b>. Code examples are given only as <i>examples</i>: the code of examples is added with new details at each step of the article. For example, at the very beginning of the article there is no error handling, "classic" methods are used (such as using <code>GetLastError</code>, the <code>sdtcall</code> convention, etc.), which are replaced by more adequate ones in the course of the article. This is done for the reason that "new" ("unusual") designs do not raise questions. Otherwise, with each example, one would have to insert a note like: "this is discussed in that paragraph below, but that - in this one here." In any case, at the end of the article there is a link to the sample code written by taking into account everything said in the article. You can just grab it and use it. And the article explains why it is created the way it is. If you are not interested in "why" - scroll to the end to the conclusion and find a link to download the example.<br />
<a name='more'></a><br /><br />
<h1>Content</h1>
<ul>
<li><a href="#t1">General Concepts</a></li>
<li><a href="#t2">Data Types</a></li>
<li><a href="#t3">String Data and Encodings</a>
<ul><li><a href="#t3_1">ANSI and Unicode</a></li></ul></li>
<li><a href="#t4">Shared Memory Manager (and why you shouldn't use it)</a></li>
<li><a href="#t5">API Memory Management</a><ul>
<li><a href="#t5_1">Incorrect Way</a></li>
<li><a href="#t5_2">Strings</a></li>
<li><a href="#t5_3">System Memory Manager</a></li>
<li><a href="#t5_4">Dedicated Wrapper Functions</a></li>
<li><a href="#t5_5">Interfaces</a></li>
</ul></li>
<li><a href="#t6">Error Handling (and calling convention)</a><ul>
<li><a href="#t6_1">Error Codes (and why you shouldn't use them)</a></li>
<li><a href="#t6_2">Exceptions (and why you shouldn't use them)</a></li>
<li><a href="#t6_3">What can and should be used (and which calling convention to use)</a>
<ul><li><a href="#t6_3_1">How to work with <code>safecall</code></a></li></ul></li>
</ul></li>
</li>
<li><a href="#t7">DllMain Workaround</a></li>
<li><a href="#t8">Callback Functions</a></li>
<li><a href="#t9">Other Rules</a></li>
<li><a href="#t10">Conclusion (and code examples)</a></li>
</ul>
<br />
<br />
<a name="t1"></a><h1>General Concepts</h1>
When you develop your own DLL, you must come up with the prototypes of the functions exported from it (i.e. "headers"), as well as the contract based on them (i.e. calling rules). Together, this forms your DLL's <b>API</b>. API or Application Programming Interface (Application Programming Interface) is a description of the ways in which one code can interact with another. In other words, API is a tool for application integration.<br />
<br />
When you develop your DLL, you must determine under what conditions it will be used:
<ol>
<li>Can it be used by applications written in another programming language (for example, Microsoft Visual C++) - "universal DLL";</li>
<li>Or the DLL can only be used by applications written in the same language, "Delphi DLL".</li>
</ol>
This is a fundamental point that you should decide first of all: even before you start writing code and even designing the API of your DLL. The fact is that you can use all the features of your programming language when creating an API in the second case ("Delphi DLL"). For example, for Delphi this means the ability to use strings, objects (in particular, forms, components), dynamic arrays, specialized simple types (<code>Extended</code>, sets, etc.) - in general, all that does not exist in other languages. It also means the ability to share memory, do transparent error handling (cross-module exceptions).<br />
<br />
If you go this route, then you should consider using run-time packages (BPLs) instead of DLLs. BPL packages are specialized DLLs that are specifically tailored for use only in Delphi, which gives you a lot of goodies. But more on that later.<br />
<br />
On the other hand, if you're developing a "generic DLL" then you can't use features in your language that don't exist in other programming languages. And in this case, you can only use "well-known" data types. But more on that below.<br />
<br />
<b>This article is mainly about "generic DLLs" in Windows.</b><br />
<br />
What you will need to create when developing your DLL API:
<ol>
<li>Headers, header files - a set of source files that contain declarations of structures and functions used in the API. As a rule, they do not contain implementations. Header files are provided in several languages - as a rule, this is the language in which the program is written (in our case, Delphi), C++ (as a standard) and some additional ones (Basic, etc.). All of these files are equivalent and are simply a translation from one programming language to another. The more languages included, the better. If you do not provide header files for a particular language, programmers in that language will not be able to use your DLL unless they themselves translate the files from the provided language (Delphi or C++) into their language. Therefore, the absence of headings in some language is not a red "stop", but a sufficient obstacle.</li>
<li>Documentation - is a verbal description of the API and it should specify additional rules not included in the header syntax. For example, the "specific function can be called by passing a number to it" fact is an information from the headers. And "before calling this function you need to call another function" fact is an information from the documentation. Such documentation should at least contain a formal description of the API - a listing of all functions, methods, interfaces and data types with explanations of "how" and "why" (the so-called Reference). Additionally, the documentation may contain an informal description of the process of using the DLL (guide, how-to, etc.). In the simplest cases, documentation is written directly in headers (comments), but most often it is a standalone file (or files) in chm, html or pdf format.</li>
</ol>
SDK (Software Development Kit) - a set of headers + documentation. The SDK is what a third party developer needs to use your DLL. An SDK is something you must create and publicly distribute to anyone who wants to use your DLL.<br />
<br />
<br />
<a name="t2"></a><h1>Data Types</h1>
If you want a "generic DLL", then you can't use Delphi-specific data types in your API because they don't have a counterpart in other languages. For example, <code>string</code>, <code>array of</code>, <code>TObject</code>, <code>TForm</code> (and in general - any objects, and even more so components) and so on. <br />
<br />
What can be used? Integer types (<code>Integer</code>, <code>Cardinal</code>, <code>Int64</code>, <code>UInt64</code>, <code>NativeInt</code>, <code >NativeUInt</code>, <code>Byte</code>, <code>Word</code> etc., I wouldn't recommend using <code>Currency</code> unless you really need it), real types (<code>Single</code> and <code>Double</code>; I would recommend avoiding the <code>Extended</code> and <code>Comp</code> types unless you really need them), <code>TDateTime</code> (it is alias for <a title="MSDN: Date Type" href="https://learn.microsoft.com/en-us/cpp/atl-mfc-shared/date-type?view=msvc-170">system's <code>OLEDate</code></a>), enumerated and subrange types (with some caveats), character types (<code>AnsiChar</code> and <code>WideChar</code> - but not the <code>Char</code>), strings (only as <code>WideString</code>/<code>BSTR</code>), boolean (<code>BOOL</code>, but not the <code>Boolean</code>), interfaces (<code>interface</code>) whose methods use valid types, records (<code>record</code>) from the above types, and pointers to them (including pointers to arrays of the above types, but not dynamic arrays). Arrays are usually passed as two parameters: a pointer to the first element of the array and the number of elements in the array.<br />
<br />
How do you know which type can be used and which can't? A relatively simple rule - if you don't see a type in this list, and the type is not in the <code>Windows</code> unit (<code>Winapi.Windows</code> unit since Delphi XE2), then that type cannot be used. If the type is listed above or it is in the <code>Windows</code>/<code>Winapi.Windows</code> unit, use it. This is a rather rough rule, but it will do for a start.<br />
<br />
In case of using records (<code>record</code>) - you need to specify data alignment. Use either the <code>packed</code> keyword (no alignment) or the {$A8} directive (8-byte alignment) at the beginning of the header file.<br />
<br />
In case of using enumerated types (<code>Color = (clRed, clBlue, clBlack);</code>) - add the <code>{$MINENUMSIZE 4}</code> directive to the beginning of the headers (the size of the enumerated type is at least 4 bytes ).<br />
<br />
<br />
<a name="t3"></a><h1>String Data and Encodings</h1>
If you need to pass strings to the DLL or return strings from the DLL - <b>use only the <code>BSTR</code></b> type. Why?
<ol>
<li>The <code>BSTR</code> type <a href="https://docs.microsoft.com/en-us/previous-versions/windows/desktop/automat/string-manipulation-functions" title=" MSDN: String Manipulation Functions">available in all programming languages</a>.<br />
Note: For historical reasons, the <code>BSTR</code> type is called <code>WideString</code> in Delphi. Therefore, to make the contents of your Delphi headers more understandable to developers in other languages, add the following code to their beginning:<br />
<pre class="brush:delphi">type
BSTR = WideString;</pre>
and then use the <code>BSTR</code> everywhere in API/headers.</li>
<li>The <code>BSTR</code> (<code>WideString</code>) type is one of Delphi's automagic types, i.e. you don't have to manually allocate and deallocate memory. The compiler will automatically do everything for you;</li>
<li>The <code>BSTR</code> type has a fixed encoding: Unicode. Therefore you won't have problems with a wrong code page;</li>
<li>Delphi compiler magic allows you to simply assign <code>BSTR</code> (via the assignment operator <code>:=</code>) to any Delphi string and vice versa. All necessary conversions will be done automatically under the hood of the language, no conversion functions need to be called;</li>
<li>Memory for <code>BSTR</code> strings <a href="https://docs.microsoft.com/en-us/cpp/atl-mfc-shared/allocating-and-releasing-memory-for-a-bstr" title="MSDN: Allocating and Releasing Memory for a BSTR">is always allocated through the same memory manager</a> so you will never have problems transferring memory between executable modules (see below);</li>
</ol>
<br />
If for some reason you can't use the <code>BSTR</code> type then use <code>PWideChar</code>:
<ol>
<li>Don't use <code>PAnsiChar</code> because it's 2023, not 1995. Using <code>PAnsiChar</code> gives you a lot of encoding headaches;</li>
<li>Do not use <code>PChar</code> because it is not uniquely defined: it can be either <code>PAnsiChar</code> or <code>PWideChar</code> (depending on compiler version).</li>
</ol>
Similar to using the system's <code>BSTR</code> name instead of the Delphi's <code>WideString</code> name: you can also do this for the <code>PWideChar</code> type:<br />
<pre class="brush:delphi">type
LPWSTR = PWideChar;</pre>
and then use <code>LPWSTR</code>. The <code>LPWSTR</code> is the name of the system data type, which is called <code>PWideChar</code> in Delphi.<br />
<br />
Of course, you get a bunch of cons when using <code>LPWSTR</code>/<code>PWideChar</code> instead of <code>WideString</code>:
<ol>
<li>You need to manually allocate and deallocate memory for <code>PWideChar</code>, which increases the chances of memory leak problems;</li>
<li>While in <i>some</i> cases you can make direct assignments (e.g. <code>PWideChar</code> to a string), more often you can't. You will have to call conversion functions and/or memory allocation/copy functions;</li>
<li>Memory for <code>PWideChar</code> strings is allocated as usual (without a dedicated memory manager), i.e. you have a problem with passing memory across a module boundary (see below);</li>
<li><code>PWideChar</code> has no field for length. So if you want to pass strings with <code>#0</code> inside and/or you want to pass large strings, then you have to explicitly pass the length of the string along with the string (two parameters instead of one).</li>
</ol>
Read more: <a href="http://rvelthuis.de/articles/articles-pchars.html" title="PChars: no strings attached">String and PChar</a>.<br / >
<br />
<a name="t3_1"></a><h2>ANSI and Unicode</h2>
From the above, it directly follows that all your exported functions must be in Unicode. Do not just look at the Windows API to make two versions of functions (with -A and -W suffixes) - just make one version (no suffix, just Unicode). Yes, do that even if you are developing on the ANSI version of Delphi (like Delphi 7): you don't need to make ANSI versions of the exported functions. It's not 1995 now.<br />
<br />
<br />
<a name="t4"></a><h1>Shared Memory Manager</h1>
<h2>(and why you shouldn't use it)</h2>
In programming languages, dynamic memory is allocated and deallocated by a special code in the program - the so-called memory manager. For example, the memory manager implements functions like <code>GetMem</code> and <code>FreeMem</code> in Delphi. All other memory management methods (<code>New</code>, <code>SetLength</code>, <code>TForm.Create</code>, etc.) are just adapters (i.e. somewhere internally they call <code>GetMem</code> and <code>FreeMem</code>).<br />
<br />
The problem is that each executable module (be it a DLL or an exe) has its own memory manager code, and, for example, the Delphi memory manager does not know anything about the Microsoft C++ memory manager (and vice versa). Therefore, if you allocate memory in Delphi and, for example, try to transfer it to Visual C++ code, then nothing good will happen. Moreover, even if you allocate memory in Delphi DLL and return it to Delphi exe, things will be even worse: both executable modules use two <i>different</i>, but <i>same type</i> memory managers. The exe memory manager will look at the memory and it will seem to him that this is his memory (after all, it is allocated by a similar memory manager), he will try to free it, but only damage the accounting data.<br />
<br />
<b>The solution to this problem is simple - you need to use the rule: whoever allocates memory, frees it.</b><br />
<br />
This rule can be enforced in a number of ways. Often mentioned method: using the so-called shared memory manager. The essence of the method is that several modules "agree" to use the same memory manager.<br />
<br />
When you create a DLL, you are told about this feature by a comment at the beginning of the .dpr file of the DLL:
<pre class="brush:delphi">{ Important note about DLL memory management: ShareMem must be the
first unit in your library's USES clause AND your project's (select
Project-View Source) USES clause if your DLL exports any procedures or
functions that pass strings as parameters or function results. This
applies to all strings passed to and from your DLL--even those that
are nested in records and classes. ShareMem is the interface unit to
the BORLNDMM.DLL shared memory manager, which must be deployed along
with your DLL. To avoid using BORLNDMM.DLL, pass string information
using PChar or ShortString parameters. }</pre>
This is a monstrously wrong comment:
<ol>
<li>The comment talks about the need to use a shared memory manager as if it is the only way to solve the memory sharing problem - which is fundamentally wrong (see below);</li>
<li>The comment only talks about strings, although the problem described applies to any data with dynamic memory allocation: objects, dynamic arrays, pointers;</li>
<li>The comment does not mention in any way what to do with non-string data;</li>
<li>Using a shared memory manager does not correlate in any way with using a standalone DLL. This is just one of the possible implementations;</li>
<li>The comment requires the use of the <code>PChar</code> type to avoid the described problem - which is also wrong (see our discussion above about encodings);</li>
<li>The comment requires the use of the <code>ShortString</code> - which, again, is wrong from a "generic DLL" point of view (<code>ShortString</code> is a Delphi-specific type). Although, this is already a nitpick, since the use of Delphi strings and Delphi DLL as a common memory manager already puts an end to the "universal DLL" concept.</li>
</ol>
Unfortunately, this comment "from the creators of Delphi" has spawned a huge amount of myths and bad practices.<br />
<br />
What's wrong with using a shared memory manager?<br />
<ol>
<li>Other programming languages know nothing about Delphi's memory manager;</li>
<li>And since you're targeting only Delphi, why do you need a DLL? Build the program with run-time packages (BPL) - this will automatically give you:
<ul>
<li>Shared memory manager in <code>rtl.bpl</code>;</li>
<li>Guaranteeing the compatibility of the structure of objects, since all modules will be assembled by one compiler;</li>
<li>No duplication of RTL and VCL (errors like "<code>TForm</code> is not compatible with <code>TForm</code>", two <code>Application</code> objects, etc.); </li>
<li>Easy error handling with exceptions.</li>
</ul>
</li>
<li>The shared memory manager makes it very difficult to find memory leaks, because a module can load, allocate memory, unload, and the created leak will only be found during the finalization of the memory manager when the program exits.</li>
</ol>
To summarize: a shared memory manager is a crutch. You don't have to use it. What should be used? See sections below.<br />
<br />
<br />
<a name="t5"></a><h1>API Memory Management</h1>
So how do you transfer memory from the DLL to the caller and vice versa? There are several ways.<br />
<br />
<a name="t5_1"></a><h2>Incorrect Way</h2>
First, what not to do.<br />
<br />
First, don't "do it like Delphi": don't use a shared memory manager - for the reasons mentioned above.<br />
<br />
Secondly, don't "do it like Windows": many look at the Windows API and do the same. But they miss the fact that this API was created in 1995, and many functions come from even earlier: 16-bit Windows. Those environments and conditions for which these functions were created no longer exist today. Today there are much simpler and more convenient ways.<br />
<br />
For example, here is a typical Windows function:<br />
<pre class="brush:delphi">function GetUserName(lpBuffer: PWideChar; var nSize: DWORD): BOOL; stdcall;</pre>
<blockquote><h3>Parameters</h3>
<b>lpBuffer</b><br />
A pointer to the buffer to receive the user's logon name. If this buffer is not large enough to contain the entire user name, the function fails.<br />
<br />
<b>pcbBuffer</b><br />
On input, this variable specifies the size of the lpBuffer buffer, in TCHARs. On output, the variable receives the number of TCHARs copied to the buffer, including the terminating null character.<br />
<br />
If lpBuffer is too small, the function fails and GetLastError returns ERROR_INSUFFICIENT_BUFFER. This parameter receives the required buffer size, including the terminating null character.</blockquote>
To get a result from such a Windows function, it must be called <b>twice</b>. First you call the function to determine the size of the buffer, then you allocate the buffer, and only then you call the function again. But what if the data changes during this time? Function may run out of space again (on the second call). Thus, to reliably get the complete data, you have to write a loop. This is horror. Don't do that.<br />
<br />
<a name="t5_2"></a><h2>Strings</h2>
Strings are easy - just use the <code>BSTR</code> (which is <code>WideString</code>). We have discussed this above in details.<br />
<br />
Note that in some cases you can return complex structured data (objects) as JSON or a similar way of packing the data into a string. And if this is your case - you can also use the <code>BSTR</code> type.<br />
<br />
In all other cases, you need to use one of the three methods below.<br />
<br />
<a name="t5_3"></a><h2>System Memory Manager</h2>
You can fulfill the "who allocates memory - releases it" rule as follows: ask a third party to allocate and release memory, which both the called and the caller know about. For example, such a third party could be any system memory manager. This is exactly how <code>BSTR</code>/<code>WideString</code> works. Here are some options you can use:
<ol>
<li>Process system heap:
<ul>
<li>The <a title="MSDN: HeapAlloc Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/heapapi/nf-heapapi-heapalloc"><code>HeapAlloc</code></a> and <a title="MSDN: HeapFree" href="https://docs.microsoft.com/en-us/windows/desktop/api/heapapi/nf-heapapi-heapfree"><code>HeapFree</code></a> called for the <a title="MSDN: GetProcessHeap Function" href="https://learn.microsoft.com/en-us/windows/win32/api/heapapi/nf-heapapi-getprocessheap"><code>GetProcessHeap</code></a>;</li>
<li>The <a title="MSDN: GlobalAlloc Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-globalalloc"><code>GlobalAlloc</code></a> and <a title="MSDN: GlobalFree Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-globalfree"><code>GlobalFree</code></a>;</li>
<li>The <a title="MSDN: LocalAlloc Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-localalloc"><code>LocalAlloc</code></a> and <a title="MSDN: LocalFree Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-localfree"><code>LocalFree</code></a>.</li>
</ul>
All of these functions allocate memory from the same process dynamic heap. Several variants appeared <a href="https://devblogs.microsoft.com/oldnewthing/20041101-00/?p=37433" title="What was the difference between LocalAlloc and GlobalAlloc?">for historical reasons</a>.</li>
<li>COM-like memory managers:
<ul>
<li>The <a title="MSND: CoTaskMemAlloc Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/combaseapi/nf-combaseapi-cotaskmemalloc"><code>CoTaskMemAlloc</code></a> and <a title="MSDN: CoTaskMemFree Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/combaseapi/nf-combaseapi-cotaskmemfree"><code>CoTaskMemFree</code></a>;</li>
<li>The <a title="MSND: SHAlloc Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/shlobj_core/nf-shlobj_core-shalloc"><code>SHAlloc</code></a> and <a title="MSDN: SHFree Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/shlobj_core/nf-shlobj_core-shfree"><code>SHFree</code></a>;</li>
</ul>
And again: all these functions are equivalent today. Several feature options appeared <a href="https://devblogs.microsoft.com/oldnewthing/20040705-00/?p=38573" title="What's the difference between SHGetMalloc, SHAlloc, CoGetMalloc, and CoTaskMemAlloc">for historical reasons</a>.</li>
<li>The <a title="MSDN: VirtualAlloc Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/memoryapi/nf-memoryapi-virtualalloc"><code>VirtualAlloc</code></a> and <a title="MSDN: VirtualFree Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/memoryapi/nf-memoryapi-virtualfree"><code>VirtualFree</code></a>.</li>
</ol>
Note: COM memory manager and Shell functions can be called immediately, without COM/OLE initialization.<br />
<br />
Pretty big list. Which one is better to use?<br />
<ul>
<li>The <code>VirtualAlloc</code>/<code>VirtualFree</code> allocate memory <a title="Why is address space allocation granularity 64KB?" href="https://devblogs.microsoft.com/oldnewthing/20031008-00/?p=42223">with a granularity of 64Kb</a>, so you should only use them if you need to exchange huge data;</li >
<li><code>GlobalAlloc</code>/<code>GlobalFree</code> and <code>LocalAlloc</code>/<code>LocalFree</code> are pretty much outdated and have more overhead than <code>HeapAlloc</code>/<code>HeapFree</code>, so you don't need to use them;</li>
</ul>
That leaves us with the <code>HeapAlloc</code>/<code>HeapFree</code> and COM. The Heap option may very well be the default. COM memory managers may be more familiar to some programming languages. In addition, there is a ready-made memory manager interface (see below). In general, it is <a title="Allocating and freeing memory across module boundaries" href="https://devblogs.microsoft.com/oldnewthing/20060915-04/?p=29723">more like taste choice, not much real difference</a>.<br />
<br />
Here is an example of how it might look in code. In DLL (simplified code without error handling):<br />
<pre class="brush:delphi">uses
ActiveX; // or uses OLE2;
function GetDynData(const AFlags: DWORD; out AData: Pointer; out ADataSize: DWORD): BOOL; stdcall;
var
P: array of Something;
begin
P := { ... prepare data to return ... };
ADataSize := Length(P) * SizeOf(Something);
AData := CoTaskMemAlloc(ADataSize);
Move(Pointer(P)^, AData^, ADataSize);
Result := True;
end;</pre>
exe:
<pre class="brush:delphi">uses
ActiveX; // or uses OLE2;
var
P: array of Something;
Data: Pointer;
DataSize: DWORD;
begin
GetDynData(0, Data, DataSize);
SetLength(P, DataSize div SizeOf(Something));
Move(Data^, Pointer(P)^, DataSize);
CoTaskMemFree(Data);
// Work with P
end;</pre>
Note: it is just an example. In real applications, you can (on the callee side) both prepare data immediately in the returned buffer (provided that you know its size in advance), and (on the caller side) work with the returned data directly, without copying it to another type of buffer. <br />
<br />
Of course, at the same time, your SDK should have documentation on the <code>GetDynData</code> function, which will explicitly say that the returned memory must be freed by calling <code>CoTaskMemFree</code>, like this: <br />
<blockquote>
<h1>
<span lang="EN-US">GetDynData<o:p></o:p></span></h1>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Returns <i>XYZ</i>.</span><span lang="EN-US" style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-ansi-language: EN-US; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Syntax</span><span lang="EN-US" style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-ansi-language: EN-US; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;"><o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt; word-break: break-all;">
<span lang="EN-US" style="font-family: Consolas; font-size: 10pt;"><b>function</b> GetDynData(<o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt; word-break: break-all;">
<span lang="EN-US" style="font-family: Consolas; font-size: 10pt;"> <b>const</b>
AFlags: DWORD; <o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt; word-break: break-all;">
<span lang="EN-US" style="font-family: Consolas; font-size: 10pt;"> <b>out</b>
AData: Pointer;<o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt; word-break: break-all;">
<span lang="EN-US" style="font-family: Consolas; font-size: 10pt;"> <b>out</b>
ADataSize: DWORD<o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt; word-break: break-all;">
<span lang="EN-US" style="font-family: Consolas; font-size: 10pt;">): BOOL; <b>stdcall</b>;<o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt; word-break: break-all;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Parameters</span><span lang="EN-US" style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-ansi-language: EN-US; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;"><o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<i><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">AFlags</span></i><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;"> [</span><span style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">in, optional</span><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">]<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Type</span><span lang="EN-US" style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-ansi-language: EN-US; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">: <b>DWORD</b><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Optional flags: ...<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<i><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">AData</span></i><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;"> [</span><span style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">out</span><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">]<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Type</span><span lang="EN-US" style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-ansi-language: EN-US; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">: <b>Pointer</b><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Pointer to a data of the <code>ADataSize</code> bytes. The caller should free this data by calling the <code>CoTaskMemFree</code> function.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<i><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">ADataSize</span></i><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;"> [</span><span style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">out</span><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">]<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Type</span><span lang="EN-US" style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-ansi-language: EN-US; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">: <b>DWORD</b><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Size of the <code>AData</code> in bytes.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Returns</span><span lang="EN-US" style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-ansi-language: EN-US; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">If the function succeeds - it returns <code>True</code>.<br />If the function fails - it returns <code>False</code>. You can call the <code>GetLastError</code> function to learn the failure reason.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Remarks</span><span lang="EN-US" style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-ansi-language: EN-US; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">...<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Examples<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">A usage example can be found in the <a href="http://www.example.com/">Getting the data sample</a> example code.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Requirements<o:p></o:p></span></div>
<table border="0" cellpadding="0" cellspacing="0" class="MsoNormalTable" style="border-collapse: collapse; mso-padding-bottom-alt: 15.0pt; mso-padding-top-alt: 15.0pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr>
<td style="background: #EDEDED; border-bottom: solid #DBDBDB 1.0pt; border: none; mso-border-bottom-alt: solid #DBDBDB .75pt; padding: 7.5pt 6.0pt 7.5pt 6.0pt;">
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 7.5pt; margin-right: 7.5pt; margin-top: 0cm;">
<b><span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">API version<o:p></o:p></span></b></div>
</td>
<td style="border-bottom: solid #DBDBDB 1.0pt; border: none; mso-border-bottom-alt: solid #DBDBDB .75pt; padding: 7.5pt 6.0pt 7.5pt 6.0pt;" valign="top">
<div class="MsoNormal" style="margin-bottom: 7.5pt; margin-left: 7.5pt; margin-right: 7.5pt; margin-top: 7.5pt;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">1<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="background: #EDEDED; border-bottom: solid #DBDBDB 1.0pt; border: none; mso-border-bottom-alt: solid #DBDBDB .75pt; padding: 7.5pt 6.0pt 7.5pt 6.0pt;">
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 7.5pt; margin-right: 7.5pt; margin-top: 0cm;">
<b><span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Headers<o:p></o:p></span></b></div>
</td>
<td style="border-bottom: solid #DBDBDB 1.0pt; border: none; mso-border-bottom-alt: solid #DBDBDB .75pt; padding: 7.5pt 6.0pt 7.5pt 6.0pt;" valign="top">
<div class="MsoNormal" style="margin-bottom: 0.0001pt; margin-left: 7.5pt; margin-right: 7.5pt; margin-top: 0cm;">
<span lang="EN-US" style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-ansi-language: EN-US; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">MyDll.pas<o:p></o:p></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<br /></div>
</blockquote>
<br />
Note: of course, the <code>CoTaskMemAlloc</code>/<code>CoTaskMemFree</code> calls can be replaced with <code>HeapAlloc</code>/<code>HeapFree</code> or any other option convenient for you. <br />
<br />
Note that with this method, you typically need to copy the data twice: in the callee (to copy the data from the prepared location to a location suitable for return to the caller) and possibly in the caller (to copy the returned data into structures suitable for further use). Sometimes you can get away with a single copy if the caller can use the data right away. But it is rare to get rid of copying data in the callee.<br />
<br />
<a name="t5_4"></a><h2>Dedicated Wrapper Functions</h2>
Another option is to wrap your preferred memory manager in an exportable function. Accordingly, the documentation for the function should indicate that to free the returned memory, you need to call not <code>CoTaskMemFree</code> (or whatever you used there), but your wrapper function. Then you can simply return prepared data immediately, without copying. For example, in a DLL (simplified code without error handling):<br />
<pre class="brush:delphi">function GetDynData(const AFlags: DWORD; out AData: Pointer; out ADataSize: DWORD): BOOL; stdcall;
var
P: array of Something;
begin
P := { ... prepare your data ... };
ADataSize := Length(P) * SizeOf(Something);
Pointer(AData) := Pointer(P); // copy the pointer, not the data itself
Pointer(P) := nil; // block the auto-release
Result := True;
end;
procedure DynDataFree(var AData: Pointer); stdcall;
var
P: array of Something;
begin
if AData = nil then
Exit;
Pointer(P) := Pointer(AData); // and again: copy just the pointer
AData := nil;
Finalize(P); // a matching release function
// (it is optional in this particular case)
end;</pre>
exe:
<pre class="brush:delphi">var
P: array of Something;
Data: Pointer;
DataSize: DWORD;
begin
GetDynData(0, Data, DataSize);
SetLength(P, DataSize div SizeOf(Something));
Move(Data^, Pointer(P)^, DataSize);
DynDataFree(Data);
// Work with P
end;</pre>
Note: we can't just copy the pointer to the array on the caller's side because the <code>GetDynData</code> contract says nothing about the compatibility of the returned data with Delphi's dynamic array. Indeed, a DLL can be written in MS Visual C++, which does not have dynamic arrays.<br />
<br />
As in the previous case, this contract must also be explicitly stated in your SDK's documentation:<br />
<blockquote><div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<i><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">AData</span></i><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;"> [</span><span style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">out</span><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">]<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Type</span><span lang="EN-US" style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-ansi-language: EN-US; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">: <b>Pointer</b><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Pointer to a data of the <code>ADataSize</code> bytes. The caller should free this data by calling the <code>DynDataFree</code> function.<o:p></o:p></span></div></blockquote>
Note that by using a wrapper function you can reduce the amount of data copying, because now you don't need to copy the data on the caller's side, because you use the same memory manager for both calculations and for returning the data. The disadvantage of this method is the need to write additional wrapper functions. Sometimes you can get away with one generic wrapper function common to all exported functions. But more often than not, you will need an individual cleanup function for each exported function (returning data) if you want to use "just one copy" advantage.<br />
<br />
If you use one generic cleanup function, you can return it as the <a href="https://docs.microsoft.com/en-us/windows/desktop/api/objidl/nn-objidl-imalloc" title="MSDN: IMalloc Interface"><code>IMalloc</code></a> interface. This will be more familiar to those familiar with the basics of COM. But it will also allow you not only to return memory to the caller, but also to accept memory from the caller with transfer of ownership. For example:<br />
<pre class="brush:delphi">uses
ActiveX; // or Ole2
type
TAllocator = class(TInterfacedObject, IMalloc)
function Alloc(cb: Longint): Pointer; stdcall;
function Realloc(pv: Pointer; cb: Longint): Pointer; stdcall;
procedure Free(pv: Pointer); stdcall;
function GetSize(pv: Pointer): Longint; stdcall;
function DidAlloc(pv: Pointer): Integer; stdcall;
procedure HeapMinimize; stdcall;
end;
{ TAllocator }
function TAllocator.Alloc(cb: Integer): Pointer;
begin
Result := AllocMem(cb);
end;
function TAllocator.Realloc(pv: Pointer; cb: Integer): Pointer;
begin
ReallocMem(pv, cb);
Result := pv;
end;
procedure TAllocator.Free(pv: Pointer);
begin
FreeMem(pv);
end;
function TAllocator.DidAlloc(pv: Pointer): Integer;
begin
Result := -1;
end;
function TAllocator.GetSize(pv: Pointer): Longint;
begin
Result := -1;
end;
procedure TAllocator.HeapMinimize;
begin
// does nothing
end;
function GetMalloc(out AAllocator: IMalloc): BOOL; stdcall;
begin
AAllocator := TAllocator.Create;
Result := True;
end;
//_______________________________________
function GetDynData(const AOptions: Pointer; out AData: Pointer; out ADataSize: DWORD): BOOL; stdcall;
var
P: array of Something;
begin
P := { ... prepare the data with AOptions ... };
// Assume the AOptions was passed with ownership
FreeMem(AOptions);
ADataSize := Length(P) * SizeOf(Something);
AData := GetMem(ADataSize);
Move(Pointer(P)^, Pointer(AData)^, ADataSize);
Result := True;
end;</pre>
exe:
<pre class="brush:delphi">var
A: IMalloc;
Options: Pointer;
P: array of Something;
Data: Pointer;
DataSize: DWORD;
begin
GetMalloc(A);
Options := A.Alloc({ options' size });
{ Preparing Options }
GetDynData(Options, Data, DataSize);
// Do not free Options, because we have passed ownership to the GetDynData
SetLength(P, DataSize div SizeOf(Something));
Move(Data^, Pointer(P)^, DataSize);
A.Free(Data);
// Work with P
end;</pre>
Note: of course, it is a bit of a nonsensical example, because in this particular case there is no need to pass the ownership of the <code>AOptions</code> to the <code>GetDynData</code> function: the caller can clean up the memory himself, then the callee will not need to free the memory. But it is just an example. In real applications, you may need to keep <code>AOptions</code> inside the DLL for longer than the function's call. The example shows how this can be done by hiding the memory manager behind a facade.<br />
<br />
Also note that if you implement the <code>TAllocator.GetSize</code> method, then the <code>ADataSize</code> parameter can be removed.<br />
<br />
<a name="t5_5"></a><h2>Interfaces</h2>
Instead of using the system memory manager and/or special export functions (two ways above), it is much more convenient to use <code>interface</code>s for the following reasons:<br />
<ol>
<li>An interface is a record with function pointers, an analogue of a class with virtual functions. Due to this, each method automatically becomes a wrapper function from the previous paragraph, i.e. always works with the right memory manager. In other words, there is no need to use a fixed third-party memory manager, nor to introduce wrapper functions;</li>
<li>Any programming languages can understand interfaces;</li>
<li>Interfaces can pass complex data (objects);</li>
<li>Interfaces are self-cleanup types (in Delphi), no need to explicitly call cleanup functions;</li>
<li>Interfaces can be easily modified by extending them in future versions of your DLL;</li>
<li>The way Delphi implements interfaces using compiler magic makes it easy to implement proper error handling (see the next section below).</li>
</ol>
The previous example can be implemented with interfaces like this:<br />
<pre class="brush:delphi">type
IData = interface
['{C79E39D8-267C-4726-98BF-FF4E93AE1D44}']
function GetData: Pointer; stdcall;
function GetDataSize: DWORD; stdcall;
property Data: Pointer read GetData;
property DataSize: DWORD read GetDataSize;
end;
TData = class(TInterfacedObject, IData)
private
FData: Pointer;
FDataSize: DWORD;
protected
function GetData: Pointer; stdcall;
function GetDataSize: DWORD; stdcall;
public
constructor Create(const AData: Pointer; const ADataSize: DWORD);
end;
constructor TData.Create(const AData: Pointer; const ADataSize: DWORD);
begin
inherited Create;
if ADataSize > 0 then
begin
GetMem(FData, ADataSize);
Move(AData^, FData^, ADataSize);
end;
end;
function TData.GetData: Pointer; stdcall;
begin
Result := FData;
end;
function TData.GetDataSize: DWORD; stdcall;
begin
Result := FDataSize;
end;
//________________________________
function GetDynData(const AFlags: DWORD; out AData: IData): BOOL; stdcall;
var
P: array of Something;
begin
P := { ... preparing the data ... };
AData := TData.Create(Pointer(P), Length(P) * SizeOf(Something));
Result := True;
end;</pre>
exe:
<pre class="brush:delphi">var
P: array of Something;
Data: IData;
begin
GetDynData(0, Data);
SetLength(P, Data.DataSize div SizeOf(Something));
Move(Data^, Data.Data^, Data.DataSize);
// Work with P
end;</pre>
In this case, we have made one universal <code>IData</code> interface that can be written once and be used in all functions. Although it does not require writing special code for every function, it also results in data copying on the side of the callee, as well as lack of typing. Here's what an improved DLL might look like:<br />
<pre class="brush:delphi">type
IData = interface
['{C79E39D8-267C-4726-98BF-FF4E93AE1D44}']
function GetData: Pointer; stdcall;
function GetDataSize: DWORD; stdcall;
property Data: Pointer read GetData;
property DataSize: DWORD read GetDataSize;
end;
TSomethingArray = array of Something;
TSomethingData = class(TInterfacedObject, IData)
private
FData: TSomethingArray;
FDataSize: DWORD;
protected
function GetData: Pointer; stdcall;
function GetDataSize: DWORD; stdcall;
public
constructor Create(var AData: TSomethingArray);
end;
constructor TSomethingData.Create(var AData: TSomethingArray);
begin
inherited Create;
FDataSize := Length(AData) * SizeOf(Something);
if FDataSize > 0 then
begin
Pointer(FData) := Pointer(AData);
Pointer(AData) := nil;
end;
end;
function TSomethingData.GetData: Pointer; stdcall;
begin
Result := Pointer(FData);
end;
function TSomethingData.GetDataSize: DWORD; stdcall;
begin
Result := FDataSize;
end;
function GetDynData(const AFlags: DWORD; out AData: IData): BOOL; stdcall;
var
P: TSomethingArray;
begin
P := { ... preparing the data ... };
AData := TSomethingData.Create(P);
Result := True;
end;</pre>
In this case, the outer wrapper (that is, the interface) remains unchanged, only the DLL code changes. So the caller's code (in the exe) doesn't change either. But if you change the contract (interface), then you can do the following:<br />
<pre class="brush:delphi">type
ISomethingData = interface
['{CF8DF791-1E8D-4363-94A2-9FF035A9015A}']
function GetData: Pointer; stdcall;
function GetDataSize: DWORD; stdcall;
function GetCount: Integer; stdcall;
function GetItem(const AIndex: Integer): Something; stdcall;
property Data: Pointer read GetData;
property DataSize: DWORD read GetDataSize;
property Count: Integer read GetCount;
property Items[const AIndex: Integer]: Something read GetItem; default;
end;
TSomethingArray = array of Something;
TSomethingData = class(TInterfacedObject, ISomethingData)
private
FData: TSomethingArray;
FDataSize: DWORD;
protected
function GetData: Pointer; stdcall;
function GetDataSize: DWORD; stdcall;
function GetCount: Integer; stdcall;
function GetItem(const AIndex: Integer): Something; stdcall;
public
constructor Create(var AData: TSomethingArray);
end;
constructor TSomethingData.Create(var AData: TSomethingArray);
begin
inherited Create;
FDataSize := Length(AData) * SizeOf(Something);
if FDataSize > 0 then
begin
Pointer(FData) := Pointer(AData);
Pointer(AData) := nil;
end;
end;
function TSomethingData.GetData: Pointer; stdcall;
begin
Result := Pointer(FData);
end;
function TSomethingData.GetDataSize: DWORD; stdcall;
begin
Result := FDataSize;
end;
function TSomethingData.GetCount: Integer; stdcall;
begin
Result := Length(FData);
end;
function TSomethingData.GetItem(const AIndex: Integer): Something; stdcall;
begin
Result := FData[AIndex];
end;
function GetDynData(const AFlags: DWORD; out AData: ISomethingData): BOOL; stdcall;
var
P: TSomethingArray;
begin
P := { ... preparing the data ... };
AData := TSomethingData.Create(P);
Result := True;
end;</pre>
and then the caller turns into this:
<pre class="brush:delphi">var
Data: ISomethingData;
begin
GetDynData(0, Data);
// No need to copy, just work with Data:
for X := 0 to Data.Count do
AddToList(Data[X]);
end;</pre>
In general, there are quite wide possibilities, you can do almost anything you want. And even if you first made a contract through <code>IData</code>, then later you can add <code>ISomethingData</code> by simply extending the interface with inheritance. However, older clients of your version 1 DLL will use <code>IData</code>, while version 2 clients may request the more convenient <code>ISomethingData</code>.<br />
<br />
As you can see from the code above, interfaces are more useful the more complex the returned data. It is very easy to return complex objects as interfaces, while returning a simple block of memory means writing a lot of code.<br />
<br />
The obvious downside is the need to write more code for interfaces, since you need a thunk object to implement the interface. But this minus is easily neutralized by the next paragraph (see "Error Handling" below). It's also partially removed if you originally need to return an object (because it doesn't require a thunk object, the returned object itself can implement the interface).<br />
<br />
Note: the code above is just an example. In real code, you need to add error handling and move the <code>IData</code>/<code>ISomethingData</code> interface definitions to separate files (your SDK headers).<br />
<br />
<br />
<a name="t6"></a><h1>Error Handling</h1>
<h2>(and calling convention)</h2>
When a programmer writes code, he determines the sequence of actions in the program, placing operators, function calls, and so on in the right order. At the same time, the implemented sequence of actions corresponds to the logic of the algorithm: first we do this, then this, and finally this. The main code corresponds to the "ideal" situation, when all files are in their places, all variables have valid values, and so on. But during the actual operation of the program, situations inevitably occur when the code written by the programmer will operate in an unacceptable (and sometimes unforeseen) environment. Such (and some other) situations are called by the generalized word "error". Therefore, the programmer must somehow determine what he will do in such situations. How will he determine the admissibility of the situation, how to react to it, etc.<br />
<br />
As a rule, the minimum blocks subject to control are a function or procedure (subroutine). Each subroutine performs a specific task. And we can expect a different level of "success" for this task: the task was successful or an error occurred during its execution. To write reliable code, we absolutely need a way to detect error situations - how do we determine that an error has occurred in a function? And responding to them is the so-called "error recovery" (i.e.: what will we do when an error occurs?). Traditionally, there are two main ways to handle errors: error codes and exceptions.<br />
<br />
<a name="t6_1"></a><h2>Error codes</h2>
<h3>(and why not to use them)</h3>
Error codes are perhaps the easiest way to respond to errors. Its essence is simple: the subroutine must return some sign of the success of the task. There are two options here: either it will return a simple sign (success/failure), or it will return the execution status (in other words - "error description"), i.e. a certain code (number) of one of several predefined situations: the function parameters are incorrectly set, the file is not found, etc. In the first case, there may be an additional function that returns the execution status of the last function called. With this approach, errors found in the function are usually passed up (to the calling function). Each function must check the results of other function calls for errors and perform appropriate processing. Most often, the processing is simply passing the error code even higher, to the "higher" calling function. For example: function A calls B, B calls C, C detects an error and returns an error code to B. B checks the return code, sees that an error occurred, and returns an error code to A. A checks the return code and issues an error message (or decides to do something else).<br />
<br />
For example, here is a typical Windows API function:<br />
<blockquote><h1>
<span lang="EN-US">RegisterClassEx<o:p></o:p></span></h1>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Registers a window class for subsequent use in calls to the <a title="MSDN: CreateWindow Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winuser/nf-winuser-createwindoww"><code>CreateWindow</code></a> or <a title="MSDN: CreateWindowEx Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winuser/nf-winuser-createwindowexw"><code>CreateWindowEx</code></a> function.</span><span lang="EN-US" style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-ansi-language: EN-US; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Syntax</span><span lang="EN-US" style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-ansi-language: EN-US; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;"><o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt; word-break: break-all;">
<span lang="EN-US" style="font-family: Consolas; font-size: 10pt;"><b>function</b> RegisterClassEx(<o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt; word-break: break-all;">
<span lang="EN-US" style="font-family: Consolas; font-size: 10pt;"> <b>const</b>
AClass: TWndClassEx<o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt; word-break: break-all;">
<span lang="EN-US" style="font-family: Consolas; font-size: 10pt;">): ATOM; <b>stdcall</b>;<o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt; word-break: break-all;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Parameters</span><span lang="EN-US" style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-ansi-language: EN-US; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;"><o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<i><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">AClass</span></i><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;"> [</span><span style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">in</span><span lang="EN-US" style="font-family: 'Segoe UI', sans-serif; font-size: 9pt;">]<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Type</span><span lang="EN-US" style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-ansi-language: EN-US; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">: <b>TWndClassEx</b><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">A pointer to a <code>WNDCLASSEX</code> structure. You must fill the structure with the appropriate class attributes before passing it to the function.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Returns</span><span lang="EN-US" style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-ansi-language: EN-US; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">If the function succeeds, the return value is a class atom that uniquely identifies the class being registered. This atom can only be used by the <a title="MSDN: CreateWindow Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winuser/nf-winuser-createwindoww"><code>CreateWindow</code></a>, <a title="MSDN: CreateWindowEx Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winuser/nf-winuser-createwindowexw"><code>CreateWindowEx</code></a>, <a title="MSDN: GetClassInfo Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winuser/nf-winuser-getclassinfow"><code>GetClassInfo</code></a>, <a href="https://docs.microsoft.com/en-us/windows/desktop/api/winuser/nf-winuser-getclassinfow" title="MSDN: GetClassInfoEx"><code>GetClassInfoEx</code></a>, <a title="MSDN: FindWindow Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winuser/nf-winuser-findwindoww"><code>FindWindow</code></a>, <a title="MSDN: FindWindowEx Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winuser/nf-winuser-findwindowexw"><code>FindWindowEx</code></a>, and <a title="MSDN: UnregisterClass Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winuser/nf-winuser-unregisterclassw"><code>UnregisterClass</code></a> functions and the <a title="MSDN: IActiveIMMap::FilterClientWindows Method" href="https://docs.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/platform-apis/aa768087(v%3Dvs.85)"><code>IActiveIMMap.FilterClientWindows</code></a> method.<br /><br />If the function fails, the return value is zero. To get extended error information, call <a title="MSDN: GetLastError Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/errhandlingapi/nf-errhandlingapi-getlasterror"><code>GetLastError</code></a>.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Remarks</span><span lang="EN-US" style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-ansi-language: EN-US; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">... cut ...<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Examples<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">For an example, see <a href="https://docs.microsoft.com/en-us/windows/desktop/winmsg/using-window-classes" title="MSDN: Using Window Classes">Using Window Classes</a>.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="color: #db7100; font-family: "Segoe UI Light","sans-serif"; font-size: 13.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Requirements<o:p></o:p></span></div>
<table border="0" cellpadding="0" cellspacing="0" class="MsoNormalTable" style="border-collapse: collapse; mso-padding-bottom-alt: 15.0pt; mso-padding-top-alt: 15.0pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr>
<td style="background: #EDEDED; border-bottom: solid #DBDBDB 1.0pt; border: none; mso-border-bottom-alt: solid #DBDBDB .75pt; padding: 7.5pt 6.0pt 7.5pt 6.0pt;">
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 7.5pt; margin-right: 7.5pt; margin-top: 0cm;">
<b><span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Minimum supported client<o:p></o:p></span></b></div>
</td>
<td style="border-bottom: solid #DBDBDB 1.0pt; border: none; mso-border-bottom-alt: solid #DBDBDB .75pt; padding: 7.5pt 6.0pt 7.5pt 6.0pt;" valign="top">
<div class="MsoNormal" style="margin-bottom: 0.0001pt; margin-left: 7.5pt; margin-right: 7.5pt; margin-top: 0cm;">
<span lang="EN-US" style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-ansi-language: EN-US; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Windows 95<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="background: #EDEDED; border-bottom: solid #DBDBDB 1.0pt; border: none; mso-border-bottom-alt: solid #DBDBDB .75pt; padding: 7.5pt 6.0pt 7.5pt 6.0pt;">
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 7.5pt; margin-right: 7.5pt; margin-top: 0cm;">
<b><span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Headers<o:p></o:p></span></b></div>
</td>
<td style="border-bottom: solid #DBDBDB 1.0pt; border: none; mso-border-bottom-alt: solid #DBDBDB .75pt; padding: 7.5pt 6.0pt 7.5pt 6.0pt;" valign="top">
<div class="MsoNormal" style="margin-bottom: 0.0001pt; margin-left: 7.5pt; margin-right: 7.5pt; margin-top: 0cm;">
<span lang="EN-US" style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-ansi-language: EN-US; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Winuser.h<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="background: #EDEDED; border-bottom: solid #DBDBDB 1.0pt; border: none; mso-border-bottom-alt: solid #DBDBDB .75pt; padding: 7.5pt 6.0pt 7.5pt 6.0pt;">
<div class="MsoNormal" style="line-height: 13.5pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 7.5pt; margin-right: 7.5pt; margin-top: 0cm;">
<b><span style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">Library<o:p></o:p></span></b></div>
</td>
<td style="border-bottom: solid #DBDBDB 1.0pt; border: none; mso-border-bottom-alt: solid #DBDBDB .75pt; padding: 7.5pt 6.0pt 7.5pt 6.0pt;" valign="top">
<div class="MsoNormal" style="margin-bottom: 0.0001pt; margin-left: 7.5pt; margin-right: 7.5pt; margin-top: 0cm;">
<span lang="EN-US" style="color: #2a2a2a; font-family: "Segoe UI","sans-serif"; font-size: 9.0pt; mso-ansi-language: EN-US; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: RU;">User32.dll<o:p></o:p></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<br /></div></blockquote>
It is a typical way to handle errors in the classic Windows API. In this case: the so-called Win32 error codes. The Win32 error code is a usual <code>DWORD</code> number. Error codes are fixed and declared in the <code>Windows</code> unit. The absence of an error is taken as <code>ERROR_SUCCESS</code> or <code>NO_ERROR</code> (equal to 0). Constants are defined for all possible errors. Those begin (usually) with the <code>ERROR_</code> word, for example:
<pre class="brush:delphi"> { Incorrect function. }
ERROR_INVALID_FUNCTION = 1; { dderror }
{ The system cannot find the file specified. }
ERROR_FILE_NOT_FOUND = 2;
{ The system cannot find the path specified. }
ERROR_PATH_NOT_FOUND = 3;
{ The system cannot open the file. }
ERROR_TOO_MANY_OPEN_FILES = 4;
{ Access is denied. }
ERROR_ACCESS_DENIED = 5;
{ The handle is invalid. }
ERROR_INVALID_HANDLE = 6;
// ... and so on</pre>
A description of the Win32 error can be obtained via the <a title="MSDN: FormatMessage Function" href="https://docs.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-formatmessage"><code>FormatMessage</code></a> function. There is (specifically for our case) a more convenient wrapper in Delphi for this system function with a bunch of parameters: the <a title="DocWiki: System.SysUtils.SysErrorMessage" href="https://docwiki.embarcadero.com/Libraries/en/System.SysUtils.SysErrorMessage"><code>SysErrorMessage</code></a> function. It returns the human-readable description of the passed Win32 error code. By the way, note that messages are returned localized. In other words, if you have Russian Windows, then the messages will be in Russian. If English - in English.<br />
<br />
Summarizing what has been said, you have to call such functions like this: <br />
<pre class="brush:delphi">{ prepare the WndClass }
ClassAtom := RegisterClassEx(WndClass);
if ClassAtom = 0 then
begin
// some error occurred, failure reason is indicated by GetLastError
Application.MessageBox(
PChar('There was an error: ' + SysErrorMessage(GetLastError)),
PChar('Error'), MB_OK or MB_ICONSTOP);
Exit;
end;
// ... continue normal execution
</pre>
<br />
As in the memory management case - it is the same: <b>do not follow the example of Windows</b>. This style has long been outdated. And here's what's wrong with him (it is not a complete list):<br />
<ol>
<li>To call a function, two calls are required: the function itself and the <code>GetLastError</code> (add to this the need to call the function itself twice to get memory from it - it turns out to be a terrible horror as much as four function calls instead of one);</li>
<li>You need to explicitly write a check like <code><b>if</b> <i>something</i> <b>then</b> <i>error</i></code>. And if you forget to write this code, you will get a bug: your program will continue executing ignoring the error. Probably corrupting the data and making it difficult to localize the bug (the visible problem will happen later);
<ul><li>If-like checks also clog the code visually;</li></ul></li>
<li>If in case of an error you need to free some resources, and even if there are several of them and there are also several function calls, then the correct code for freeing resources can become very non-trivial;</li>
<li>You can not pass any additional information. For example, you can not specify in any way which argument is incorrect, or which file you do not have access to;
<ul><li>You have no way of knowing which function failed, whether it was the function you called, or maybe some other function that the one you called might have called;</li></ul></li>
<li>The debugger will not notify you of the problem in any way (although, hypothetically, you could put a breakpoint on the <code>GetLastError</code>).</li>
</ol>
<br />
Despite all the disadvantages, error codes have a plus: since they are just numbers, they are understandable to any programming language. In other words, error codes are compatible between different languages.<br />
<br />
<a name="t6_2"></a><h2>Exceptions</h2>
<h3>(and why not to use them)</h3>
Exceptions don't have many of the disadvantages of error codes:<br />
<ol>
<li>Exceptions do not need to be explicitly checked, the default reaction is an error response;
<ul><li>The program is not "polluted" with the check code, it is taken out of your main code;</li></ul></li>
<li>Easy to release resources (via <code>try</code>-<code>finally</code>);</li>
<li>Exceptions are easy to extend, inherit, add additional fields, make nested exceptions;</li>
<li>The debugger will notify you when an exception occurs;
<ul><li>You can assign your code to diagnose exceptions (the so-called exception tracer).</li></ul></li>
</ol>
<br />
But despite all the pluses, exceptions have one significant minus, which crosses out all the pluses (in relation to the DLL API).<br />
<br />
Recall how exceptions are raised in Delphi:<br />
<pre class="brush:delphi">var
E: Exception;
begin
E := EMyExceptionClass.Create('Something');
raise E;
end;</pre>
I split the typical "<code>raise EMyExceptionClass.Create('Something');</code>" line into two to make the problem even more obvious. We create a <b>Delphi object</b> (exception) and "throw" it. And whoever wants to handle this exception does this:<br />
<pre class="brush:delphi">except
on E: EMyException do
begin
ShowMessage(E.Message);
end; // - E will be deleted here
end;</pre>
It means that the Delphi object is passed from the callee (DLL) where the exception is thrown to the caller (exe) where the exception is handled. As we learned earlier (see the Data Types section above), this is a problem. Other programming languages don't know what a Delphi object is, nor how to read it, nor how to delete it. Even Delphi itself doesn't always know this (for example, if an exception is thrown by code built on Delphi 7 but caught by code built on Delphi XE, or vice versa). Other programming languages use similar constructs: an exception is represented by an object. Accordingly, Delphi code has no idea how to work with objects in other languages.<br />
<br />
In other words, exceptions should not be used due to language incompatibilities.<br />
<br />
Corollary 1: Exceptions should not leave your DLL.<br />
<br />
Corollary 2: You must catch all exceptions in your exported functions.<br />
<br />
Corollary 3: <b>all exported functions must have the global construct <code>try</code>-<code>except</code></b>.<br />
<pre class="brush:delphi">function GetDynData(const AFlags: DWORD; out AData: IData): BOOL; stdcall;
begin
try
// ... function's code, useful payload ...
Result := True;
except
// ... exception handling ...
Result := False;
end;
end;</pre>
<br />
<a name="t6_3"></a><h2>What can and should be used</h2>
<h3>(and which calling convention to use)</h3>
If we can't use error codes and we can't use exceptions, then what should we use? Well, we need to use their combination - and here is how it looks.<br />
<br />
Delphi has built-in compiler magic that can wrap any function in a hidden <code>try</code>-<code>except</code> block with an automatic (hidden) call to the processing function. And there is a compiler magic that works the other way around: on the returned error code, it automatically raises the appropriate exception.<br />
<br />
Before we get to know this magic, we need to get acquainted with error codes in the form of the <code>HRESULT</code> type. The <code>HRESULT</code> is also a number, but now of the <code>Integer</code> type. The <code>HRESULT</code> is no longer just an error code, it consists of several parts, which we will not go into in detail, but suffice it to say that they include the error code itself (what used to be Win32 code), a sign success or failure, identifier of the error agent. Error codes typically start with the <code>E_</code> prefix (for example, <code>E_FAIL</code>, <code>E_UNEXPECTED</code>, <code>E_ABORT</code> or <code>E_ACCESSDENIED</code>), and success codes typically start with the <code>S_</code> prefix (for example, <code>S_OK</code> or <code>S_FALSE</code>). It is easy to determine the success of the <code>HRESULT</code> code by comparing it with zero: <code>HRESULT</code> error codes must be less than zero.<br />
<br />
Highlighting the success/error indicator means that now there is no need for the function to return only the success/failure (via <code>BOOL</code>), and the error code itself - through a separate function (<code>GetLastError</code>). Now the function can return both information at once, in one call:<br />
<pre class="brush:delphi">function GetDynData(const AFlags: DWORD; out AData: IData): HRESULT; stdcall;
begin
try
// ... function's code, useful payload ...
Result := S_OK;
except
// ... exception handling ...
Result := E_FAIL; // some error code
end;
end;</pre>
<br />
Along with the introduction of the <code>HRESULT</code> type, the <a href="https://docs.microsoft.com/en-us/windows/desktop/api/oaidl/nn-oaidl-ierrorinfo" title= "MSDN: IErrorInfo Interface"><code>IErrorInfo</code></a> interface was added - which allows you to associate additional information with the returned <code>HRESULT</code>: arbitrary description, GUID of the raising party (interface), the location of the error (arbitrary line), help. You don't even need to implement this interface, the system already has an object ready - returned by the <a href="https://docs.microsoft.com/en-us/windows/desktop/api/oleauto/nf-oleauto-createerrorinfo" title="MSDN: CreateErrorInfo Function"><code>CreateErrorInfo</code></a> function.<br />
<br />
Finally, Delphi has the already mentioned compiler magic that can make writing such code easier. To do this, the function must have a calling convention <code>stdcall</code> and return the <code>HRESULT</code> type. If before the function returned some <code>Result</code>, then this <code>Result</code> should be converted to the last out-parameter, for example:<br />
<pre class="brush:delphi">// Was:
function GetDynData(const AFlags: DWORD): IData;
// Became:
function GetDynData(const AFlags: DWORD; out AData: IData): HRESULT; stdcall;</pre>
If a function satisfies these requirements, then you can declare it like this:<br />
<pre class="brush:delphi">function GetDynData(const AFlags: DWORD): IData; safecall;</pre>
This would be binary equivalent (i.e. fully compatible) to:
<pre class="brush:delphi">function GetDynData(const AFlags: DWORD; out AData: IData): HRESULT; stdcall;</pre>
<br />
By declaring a function as <code>safecall</code> you turn on compiler magic for it, namely:<br />
<ol>
<li>The returned result will be automatically converted to the last out parameter;</li>
<li>The function will implicitly return the <code>HRESULT</code> (and possibly <code>IErrorInfo</code>);</li>
<li>The function's call will be wrapped in an if-check on the return code. If an erroneous <code>HRESULT</code> is received, an exception will be raised:
<pre class="brush:delphi">var
Data: IData;
begin
Data := GetDynData(Flags); // throwns exception when error occurs
// execution continues only when there is no error</pre></li>
<li>The function itself will be wrapped in a hidden <code>try</code>-<code>except</code> block that converts the exception to <code>HRESULT</code> (and possibly to <code>IErrorInfo</code>):
<pre class="brush:delphi">function GetDynData(const AFlags: DWORD): IData; safecall;
begin
// ... function's code, useful payload ...
end; // - a hidden try-except block</pre></li>
</ol>
As you can see, with this compiler support, you can write code almost as if it was a regular function in a regular Delphi unit. And the best part is that other programming languages can use a similar approach. Of course, other programming language may not have a similar compiler magic, but any language is perfectly capable of taking the <code>HRESULT</code> of the <code>stdcall</code> function and parsing it (perhaps along with <code>IErrorInfo</code>).<br />
<br />
<a name="t6_3_1"></a><h3>How to use <code>safecall</code> correctly</h3>
Now that we've covered the benefits of the <code>safecall</code>, it's time for a fly in the ointment. The fact is that the <code>safecall</code> magic works in a minimal mode "out of the box". And to get maximum benefit from it, we need to take additional steps. Luckily, they only need to be made once and can be reused in the future.<br />
<br />
Item number one: simple exported functions:<br />
<pre class="brush:delphi">procedure DoSomething; safecall;
begin
// ... function's code, useful payload ...
end;
exports
DoSomething;</pre>
Unfortunately, the compiler does not allow customizing the process of converting an exception to <code>HRESULT</code> for ordinary functions, always returning a fixed code and losing additional error information. Therefore, instead of exported functions, you need to use interfaces with methods. Before:<br />
<pre class="brush:delphi">procedure DoSomething; safecall;
begin
// ... function's code, useful payload ...
end;
function GetDynData(const AFlags: DWORD): IData; safecall;
begin
// ... function's code, useful payload ...
end;
function DoSomethingElse(AOptions: IOptions): BSTR; safecall;
begin
// ... function's code, useful payload ...
end;
exports
DoSomething,
GetDynData,
DoSomethingElse;</pre>
After:<br />
<pre class="brush:delphi">type
IMyDLL = interface
['{C5DBE4DC-B4D7-475B-9509-E43193796633}']
procedure DoSomething; safecall;
function GetDynData(const AFlags: DWORD): IData; safecall;
function DoSomethingElse(AOptions: IOptions): BSTR; safecall;
end;
TMyDLL = class(TInterfacedObject, IMyDLL)
protected
procedure DoSomething; safecall;
function GetDynData(const AFlags: DWORD): IData; safecall;
function DoSomethingElse(AOptions: IOptions): BSTR; safecall;
end;
procedure TMyDLL.DoSomething; safecall;
begin
// ... function's code, useful payload ...
end;
function TMyDLL.GetDynData(const AFlags: DWORD): IData; safecall;
begin
// ... function's code, useful payload ...
end;
function TMyDLL.DoSomethingElse(AOptions: IOptions): BSTR; safecall;
begin
// ... function's code, useful payload ...
end;
function GetFunctions(out AFunctions: IMyDLL): HRESULT; stdcall;
begin
try
AFunctions := TMyDLL.Create;
Result := S_OK;
except
on E: Exception do
Result := HandleSafeCallException(E, ExceptAddr);
end;
end;
exports
GetFunctions;</pre>
where <code>HandleSafeCallException</code> is our function, which we will describe below.<br />
<br />
As you can see, we have placed all exported functions in a single interface (object) - this will allow us to set up/control the process of converting exceptions to <code>HRESULT</code>. In this case, the DLL exports the only function that we had to write manually, without <code>safecall</code> - which also allowed us to control the conversion process. Don't forget that it is binary compatible with <code>safecall</code>, so if you want to use this DLL in Delphi you can do this:<br />
<pre class="brush:delphi">function GetFunctions: IMyDLL; safecall; external 'MyDLL.dll';</pre>
and it will work just fine.<br />
<br />
For objects, when an exception is thrown in a <code>safecall</code> method, the compiler calls the <a title="DocWiki: System.TObject.SafeCallException" href="https://docwiki.embarcadero.com/Libraries/en/System.TObject.SafeCallException"><code>TObject.SafeCallException</code></a> virtual method which does nothing useful by default and which we can replace with our own method:<br />
<pre class="brush:delphi">
type
TMyDLL = class(TInterfacedObject, IMyDLL)
protected
procedure DoSomething; safecall;
function GetDynData(const AFlags: DWORD): IData; safecall;
function DoSomethingElse(AOptions: IOptions): BSTR; safecall;
public
function SafeCallException(ExceptObject: TObject; ExceptAddr: Pointer): HResult; override;
end;
function TMyDLL.SafeCallException(ExceptObject: TObject; ExceptAddr: Pointer): HResult;
begin
Result := HandleSafeCallException(ExceptObject, ExceptAddr);
end;</pre>
<br />
Further, when the code calls a <code>safecall</code> method, the compiler wraps the method's call in a <code>CheckAutoResult</code> wrapper, which (in case of erroneous code) raises an exception through the <code>SafeCallErrorProc</code> global variable function, which, again, we can replace with our own:<br />
<pre class="brush:delphi">procedure RaiseSafeCallException(ErrorCode: HResult; ErrorAddr: Pointer);
begin
// ... our code ...
end;
initialization
SafeCallErrorProc := RaiseSafeCallException;
end.</pre>
Now we just need to make our <code>HandleSafeCallException</code> and <code>RaiseSafeCallException</code> work as a pair and do something useful.<br />
<br />
First we need two helper wrapper functions:<br />
<pre class="brush:delphi">uses
ActiveX; // or Ole2
function SetErrorInfo(const ErrorCode: HRESULT; const ErrorIID: TGUID;
const Source, Description, HelpFileName: WideString;
const HelpContext: Integer): HRESULT;
var
CreateError: ICreateErrorInfo;
ErrorInfo: IErrorInfo;
begin
Result := E_UNEXPECTED;
if Succeeded(CreateErrorInfo(CreateError)) then
begin
CreateError.SetGUID(ErrorIID);
if Source <> '' then
CreateError.SetSource(PWideChar(Source));
if HelpFileName <> '' then
CreateError.SetHelpFile(PWideChar(HelpFileName));
if Description <> '' then
CreateError.SetDescription(PWideChar(Description));
if HelpContext <> 0 then
CreateError.SetHelpContext(HelpContext);
if ErrorCode <> 0 then
Result := ErrorCode;
if CreateError.QueryInterface(IErrorInfo, ErrorInfo) = S_OK then
ActiveX.SetErrorInfo(0, ErrorInfo);
end;
end;
procedure GetErrorInfo(out ErrorIID: TGUID; out Source, Description, HelpFileName: WideString; out HelpContext: Longint);
var
ErrorInfo: IErrorInfo;
begin
if ActiveX.GetErrorInfo(0, ErrorInfo) = S_OK then
begin
ErrorInfo.GetGUID(ErrorIID);
ErrorInfo.GetSource(Source);
ErrorInfo.GetDescription(Description);
ErrorInfo.GetHelpFile(HelpFileName);
ErrorInfo.GetHelpContext(HelpContext);
end
else
begin
FillChar(ErrorIID, SizeOf(ErrorIID), 0);
Source := '';
Description := '';
HelpFileName := '';
HelpContext := 0;
end;
end;</pre>
As you can easily imagine, they are intended to pass and receive additional information along with <code>HRESULT</code>.<br />
<br />
Next, we need a way to somehow pass the class name of the exception. You can do this in different ways. For example, pass it directly to <code>HRESULT</code>. To do this, it needs to be encoded. For example, like this:<br />
<pre class="brush:delphi">uses
ComObj, // for the EOleSysError and EOleException
VarUtils; // for the ESafeArrayError
const
// ID for our DLL API rules
ThisDllIID: TGUID = '{AA76E538-EF3C-4F35-9914-B4801B211A6D}';
// "Customer" bit, it is always 0 for Microsoft-defined codes
CUSTOMER_BIT = 1 shl 29;
// Delphi uses this value to pass EAbort
// It is assumed that E_Abort should show "Aborted" message,
// while EAbortRaisedHRESULT should be handled silently.
EAbortRaisedHRESULT = HRESULT(E_ABORT or CUSTOMER_BIT);
function Exception2HRESULT(const E: TObject): HRESULT;
function NTSTATUSFromException(const E: EExternal): DWORD;
begin
// ...
end;
begin
if E = nil then
Result := E_UNEXPECTED
else
if not E.InheritsFrom(Exception) then
Result := E_UNEXPECTED
else
if E.ClassType = Exception then
Result := E_FAIL
else
if E.InheritsFrom(ESafecallException) then
Result := E_FAIL
else
if E.InheritsFrom(EAssertionFailed) then
Result := E_UNEXPECTED
else
if E.InheritsFrom(EAbort) then
Result := EAbortRaisedHRESULT
else
if E.InheritsFrom(EOutOfMemory) then
Result := E_OUTOFMEMORY
else
if E.InheritsFrom(ENotImplemented) then
Result := E_NOTIMPL
else
if E.InheritsFrom(ENotSupportedException) then
Result := E_NOINTERFACE
else
if E.InheritsFrom(EOleSysError) then
Result := EOleSysError(E).ErrorCode
else
if E.InheritsFrom(ESafeArrayError) then
Result := ESafeArrayError(E).ErrorCode
else
if E.InheritsFrom(EOSError) then
Result := HResultFromWin32(EOSError(E).ErrorCode)
else
if E.InheritsFrom(EExternal) then
if Failed(HRESULT(EExternal(E).ExceptionRecord.ExceptionCode)) then
Result := HResultFromNT(Integer(EExternal(E).ExceptionRecord.ExceptionCode))
else
Result := HResultFromNT(Integer(NTSTATUSFromException(EExternal(E))))
else
Result := MakeResult(SEVERITY_ERROR, FACILITY_ITF, Hash(E.ClassName)) or CUSTOMER_BIT;
end;</pre>
Here we are checking for a few special predefined classes, and we also have the ability to pass Win32 codes and hardware exception codes directly. For all other (Delphi specific) exception classes, we use the hash on the class name along with <code>FACILITY_ITF</code>. As a hash, you can use, for example, SDBM - this is a very simple hash function with good randomization of the result. Of course, you can use any other method - for example, just manually extract and fix the codes for each exception class.<br />
<blockquote><code>HRESULT</code>s with <code>FACILITY_NULL</code> and <code>FACILITY_RPC</code> codes have a generic value because they are defined by Microsoft. <code>HRESULT</code> with <code>FACILITY_ITF</code> code are defined by the interface function or method from which they are returned. This means that the same 32-bit value in <code>FACILITY_ITF</code> but returned by two different interfaces can have different meanings. In this way, Microsoft can define multiple generic error codes while still allowing other programmers to define new error codes without fear of conflict. The coding convention looks like this:
<ul>
<li><code>HRESULT</code> with codes other than <code>FACILITY_ITF</code> can only be defined by Microsoft;</li>
<li><code>HRESULT</code> of <code>FACILITY_ITF</code> are defined solely by the implementer of the interface or function that returns <code>HRESULT</code>. To avoid conflicting <code>HRESULT</code>, whoever defines an interface is responsible for coordinating and publishing the <code>HRESULT</code> codes associated with that interface;</li>
<li>All <code>HRESULT</code> defined by Microsoft have an error code value in the $0000-$01FF range. Although you can use any code with <code>FACILITY_ITF</code>, it is recommended to use values in the $0200-$FFFF range. This recommendation is intended to reduce confusion with Microsoft codes.</li>
</ul>
</blockquote>
That's why in the code above we also defined <code>ThisDllIID</code> which is an "interface" identifier that gives meaning to returned codes of type <code>FACILITY_ITF</code>. This value must be passed as <code>ErrorIID</code> to the <code>SetErrorInfo</code> defined above.<br />
<br />
The 29th "Customer" bit was originally a reserved bit, which was later allocated to be used as a flag indicating whether the code is defined by Microsoft (0) or by a third party (1). In a way, this bit duplicates <code>FACILITY_ITF</code>. Usually even third party developers only use <code>FACILITY_ITF</code>. In this case, we set it to reduce possible problems with bad code (which does not take into account the GUID of the interface).<br />
<br />
Wverything is a little more complicated with the reverse conversion (code to exception): we need tables to search for the exception class by code. A simple implementation might look like this:<br />
<pre class="brush:delphi">function HRESULT2Exception(const E: HRESULT): Exception;
function MapNTStatus(const ANTStatus: DWORD): ExceptClass;
begin
// ...
end;
function MapException(const ACode: DWORD): ExceptClass;
begin
// ...
end;
var
NTStatus: DWORD;
ErrorIID: TGUID;
Source: WideString;
Description: WideString;
HelpFileName: WideString;
HelpContext: Integer;
begin
if GetErrorInfo(ErrorIID, Source, Description, HelpFileName, HelpContext) then
begin
if Pointer(StrToInt64Def(Source, 0)) <> nil then
ErrorAddr := Pointer(StrToInt64(Source));
end
else
Description := SysErrorMessage(DWORD(E));
if (E = E_FAIL) or (E = E_UNEXPECTED) then
Result := Exception.Create(Description)
else
if E = EAbortRaisedHRESULT then
Result := EAbort.Create(Description)
else
if E = E_OUTOFMEMORY then
begin
OutOfMemoryError;
Result := nil;
end
else
if E = E_NOTIMPL then
Result := ENotImplemented.Create(Description)
else
if E = E_NOINTERFACE then
Result := ENotSupportedException.Create(Description)
else
if HResultFacility(E) = FACILITY_WIN32 then
begin
Result := EOSError.Create(Description);
EOSError(Result).ErrorCode := HResultCode(E);
end
else
if E and FACILITY_NT_BIT <> 0 then
begin
// Get exception's class by code
NTStatus := Cardinal(E) and (not FACILITY_NT_BIT);
Result := MapNTStatus(NTStatus).Create(Description);
// Create a dummy ExceptionRecord just in case
ReallocMem(Pointer(Result), Result.InstanceSize + SizeOf(TExceptionRecord));
EExternal(Result).ExceptionRecord := Pointer(NativeUInt(Result) + Cardinal(Result.InstanceSize));
FillChar(EExternal(Result).ExceptionRecord^, SizeOf(TExceptionRecord), 0);
EExternal(Result).ExceptionRecord.ExceptionCode := cDelphiException;
EExternal(Result).ExceptionRecord.ExceptionAddress := ErrorAddr;
end
else
if (E and CUSTOMER_BIT <> 0) and
(HResultFacility(E) = FACILITY_ITF) and
CompareMem(@ThisDllIID, @ErrorIID, SizeOf(ErrorIID)) then
Result := MapException(HResultCode(E)).Create(Description)
else
Result := EOleException.Create(Description, E, Source, HelpFileName, HelpContext);
end;</pre>
In general, the code is fairly straightforward, with the exception of hardware exceptions. We make emulation for them.<br />
<br />
Also note that the <code>Source</code> field of the <code>IErrorInfo</code> interface must point to the location where the error occurred. This field is arbitrary and is determined by the interface developer (ie, again, by GUID). In this case, we just write the address of the exception there. But, for example, if you use an exception tracer (such as EurekaLog), you can write the call stack there.<br />
<br />
Then with the above helper functions, our <code>HandleSafeCallException</code> and <code>RaiseSafeCallException</code> become trivial:<br />
<pre class="brush:delphi">function HandleSafeCallException(ExceptObj: TObject; ErrorAddr: Pointer): HRESULT;
var
ErrorMessage: String;
HelpFileName: String;
HelpContext: Integer;
begin
if ExceptObj is Exception then
ErrorMessage := Exception(ExceptObj).Message
else
ErrorMessage := SysErrorMessage(DWORD(E_FAIL));
if ExceptObj is EOleException then
begin
HelpFileName := EOleException(ExceptObj).HelpFile;
HelpContext := EOleException(ExceptObj).HelpContext;
end
else
begin
HelpFileName := '';
if ExceptObj is Exception then
HelpContext := Exception(ExceptObj).HelpContext
else
HelpContext := 0;
end;
Result := SetErrorInfo(Exception2HRESULT(ExceptObj), ThisDllIID,
'$' + IntToHex(NativeUInt(ErrorAddr), SizeOf(ErrorAddr) * 2), ErrorMessage,
HelpFileName, HelpContext);
end;
procedure RaiseSafeCallException(ErrorCode: HResult; ErrorAddr: Pointer);
var
E: Exception;
begin
E := HRESULT2Exception(ErrorCode, ErrorAddr);
raise E at ErrorAddr;
end;</pre>
Note: in our model, we do not use help fields of the <code>IErrorInfo</code> interface.<br />
<br />
It should be noted that if an interface uses <code>HRESULT</code> and <code>IErrorInfo</code> together, then it should also implement the <code>ISupportErrorInfo</code> interface. Some programming languages require this. By calling <code>ISupportErrorInfo.InterfaceSupportsErrorInfo</code>, the client side can determine that an object supports additional information.<br />
<br />
And the last point - in the Delphi implementation for Windows 32-bit there is <a title="Quality Portal: [RSP-24652] TObject.SafeCallException might be NOT called in certain cases" href="https://quality.embarcadero.com/browse/RSP-24652">nasty bug</a> that doesn't exist in 64-bit RTL, as well as on other platforms. The fix for this bug is included in the code examples at the link at the end of the article.<br />
<br />
<br />
<a name="t7"></a><h1>DllMain Workaround</h1>
The <code>DllMain</code> is a special function in a DLL that is called by the system when the DLL is loaded into, unloaded from a process (and attached/detached to/from a thread). For example, the <code>initialization</code> and <code>finalization</code> sections of your Delphi modules are executed inside <code>DllMain</code>.<br />
<br />
The problem is that <code>DllMain</code> is a very special function. It is called while holding the critical section of the loader (modules) of the operating system. In long and detailed terms - see the links at the end of this paragraph, and in short: <b><code>DllMain</code> is a weapon from which you can easily shoot yourself</b>. There are not many things that can be done legally in <code>DllMain</code>. But it's incredibly easy to do something forbidden - you constantly need to be sure that this very function that you just called can never, under any circumstances, do something forbidden. This makes it incredibly difficult to use code written elsewhere. The compiler won't tell you anything. And the code will most of the time work like it should... but sometimes it will crash or freeze.<br />
<br />
The solution to the problem is to do nothing in <code>DllMain</code> (read: don't write code in <code>initialization</code> and <code>finalization</code> sections of your units when you create a DLL).<br />
<br />
Instead, you need to <b>make separate DLL initialization and finalization functions</b>. You need to do them even if your DLL doesn't need any initialization or cleanup steps. After all, such a need may arise in the future, and if you do not provide separate initialization and finalization functions in your API, you will not be able to solve this problem later.<br />
<br />
Here is the code template:<br />
<pre class="brush:delphi">// In headers:
type
IMyDll = interface
['{C5DBE4DC-B4D7-475B-9509-E43193796633}']
procedure InitDLL(AOptional: IUnknown = nil); safecall;
procedure DoneDLL; safecall;
// ...
end;
// In DLL:
type
TInitFunc = procedure(const AOptional: IUnknown);
TDoneFunc = procedure;
TInitDoneFunc = record
Init: TInitFunc;
Done: TDoneFunc;
end;
procedure RegisterInitFunc(const AInitProc: TInitFunc; const ADoneFunc: TDoneFunc = nil);
// ...
var
GInitDoneFuncs: array of TInitDoneFunc;
procedure RegisterInitFunc(const AInitProc: TInitFunc; const ADoneFunc: TDoneFunc);
begin
SetLength(GInitDoneFuncs, Length(GInitDoneFuncs) + 1);
GInitDoneFuncs[High(GInitDoneFuncs)].Init := AInitProc;
GInitDoneFuncs[High(GInitDoneFuncs)].Done := ADoneFunc;
end;
procedure TMyDLL.InitDLL(AOptional: IUnknown); safecall;
var
X: Integer;
begin
for X := 0 to High(GInitDoneFuncs) do
if Assigned(GInitDoneFuncs[X].Init) then
GInitDoneFuncs.Init(AOptional);
end;
procedure TMyDLL.DoneDLL; safecall;
var
X: Integer;
begin
for X := 0 to High(GInitDoneFuncs) do
if Assigned(GInitDoneFuncs[X].Done) then
GInitDoneFuncs.Done;
end;
// In your units:
procedure InitUnit(const AOptional: IUnknown);
begin
// ... code from unit's initialization sections
end;
procedure DoneUnit;
begin
// ... code from unit's finalization section
end;
initialization
RegisterInitFunc(InitUnit, DoneUnit);
end;</pre>
The <code>AOptional</code> parameter is designed for possible future use. It is not used in the code above, but later (in the next version of the DLL) you can use it to pass initialization parameters. <code>IUnknown</code> is the base interface from which all other interfaces are inherited (i.e. some analogue of <code>TObject</code> for interfaces).<br />
<br />
I hope this code is clear enough. Of course, it must be distributed among different units and sections. Interface - in headers, <code>RegisterInitFunc</code> declaration - in <code>interface</code> of the common DLL module, you need to call it from the <code>initialization</code> section of other units.<br />
<br />
Of course, your SDK documentation should say that the user (client) of your DLL must call the <code>InitDLL</code> method immediately after loading your DLL with the <code>LoadLibrary</code> function and call the <code>DoneDLL</code> just before the DLL is unloaded by <code>FreeLibrary</code>:<br />
<pre class="brush:delphi">var
DLL: HMODULE;
DLLApi: IMyDll;
begin
DLL := LoadLibrary('MyDLL.dll');
Win32Check(DLL <> 0);
try
DLLApi.InitDLL(nil);
// working with DLL, for example, calling DLLApi.GetDynData
finally
DLLApi.DoneDLL;
DLLApi := nil;
FreeLibrary(DLL);
end;
end;</pre>
<br />
More information about the <code>DllMain</code>:<br />
<ol>
<li><a href="https://learn.microsoft.com/en-us/archive/blogs/oleglv/dllmain-and-life-before-birth" title="DllMain and life before birth">DllMain and life before birth</a></li>
<li><a href="https://learn.microsoft.com/en-us/archive/blogs/oleglv/dllmain-a-horror-story" title="DllMain : a horror story">DllMain : a horror story</a></li>
<li><a href="https://devblogs.microsoft.com/oldnewthing/20040127-00/?p=40873" title="Some reasons not to do anything scary in your DllMain">Some reasons not to do anything scary in your DllMain</a></li>
<li><a href="https://devblogs.microsoft.com/oldnewthing/20040128-00/?p=40853" title="Another reason not to do anything scary in your DllMain: Inadvertent deadlock">Another reason not to do anything scary in your DllMain: Inadvertent deadlock</a></li>
</ol>
<br />
<br />
<a name="t8"></a><h1>Callback Functions</h1>
A callback function - passing the executable code as one of the parameters of another code. For example, if you want to set a timer using the Windows API, you can call <a href="https://msdn.microsoft.com/en-us/library/windows/desktop/ms644906(v=vs.85).aspx" title="MSDN: SetTimer Function"><code>SetTimer</code></a> function, passing it a pointer to your own function, which will be the callback function. The system will call your function every time the timer fires:<br />
<pre class="brush:delphi">procedure MyTimerHandler(Wnd: HWND; uMsg: UINT; idEvent: UINT_PTR; dwTime: DWORD); stdcall;
begin
// Will be called after 100 ms timeout
end;
procedure TForm1.Button1Click(Sender: TObject);
begin
SetTimer(Handle, 1, 100, @MyTimerHandler);
end;</pre>
Here's another example: if you want to find all windows on the desktop, you can use the <a href="https://msdn.microsoft.com/en-us/library/windows/desktop/ms633497(v=vs.85 ).aspx" title="MSDN: EnumWindows Function"><code>EnumWindows</code></a> function:<br />
<pre class="brush:delphi">function MyEnumFunc(Wnd: HWND; lpData: LPARAM): Bool; stdcall;
begin
// Will be called for each found window
end;
procedure TForm1.Button1Click(Sender: TObject);
begin
EnumWindows(@MyEnumFunc, 0);
end;</pre>
Since the callback function usually performs the same task as the code that sets it up, it turns out that both pieces of code need to work with the same data. Therefore, the data from the setting code must somehow be passed to the callback function (or visa versa). For this purpose, so-called user parameters are provided in the callback functions: it is either a pointer or an integer (necessarily of the <code>Native(U)Int</code> type, but not just <code>(U)Int</code>), which are not used by the API itself in any way and are transparently passed to the callback function. Or (in rare cases) it can be some value that uniquely identifies the function's call.<br />
<br />
For example, <code>SetTimer</code> has <code>idEvent</code> and <code>EnumWindows</code> has <code>lpData</code>. We can use these parameters to pass arbitrary data. For example, here is how you can find all windows of a given class:<br />
<pre class="brush:delphi">type
PEnumArgs = ^TEnumArgs;
TEnumArgs = record
ClassName: String;
Windows: TStrings;
end;
function FindWindowsOfClass(Wnd: HWND; lpData: LPARAM): Bool; stdcall;
var
Args: PEnumArgs;
WndClassName, WndText: String;
begin
Args := Pointer(lpData);
SetLength(WndClassName, Length(Args.ClassName) + 2);
SetLength(WndClassName, GetClassName(Wnd, PChar(WndClassName), Length(WndClassName)));
if WndClassName = Args.ClassName then
begin
SetLength(WndText, GetWindowTextLength(Wnd) + 1);
SetLength(WndText, GetWindowText(Wnd, PChar(WndText), Length(WndText)));
Args.Windows.Add(Format('%8x : %s', [Wnd, WndText]));
end;
Result := True;
end;
procedure TForm1.Button1Click(Sender: TObject);
var
Args: TEnumArgs;
begin
// Edit can contain values like:
// 'TForm1', 'IME', 'MSTaskListWClass', 'Shell_TrayWnd', 'TTOTAL_CMD', 'Chrome_WidgetWin_1'
Args.ClassName := Edit1.Text;
Args.Windows := Memo1.Lines;
Memo1.Lines.BeginUpdate;
try
Memo1.Lines.Clear;
EnumWindows(@FindWindowsOfClass, LPARAM(@Args));
finally
Memo1.Lines.EndUpdate;
end;
end;</pre>
<br />
Note: here's another example of how not to do it - don't do it like in Windows. If you just need to get a list of something - don't make a callback, just return the list in an array (wrap it in an interface or pass it as a block of memory - as discussed above). The callback function should only be used if creating the list can take a long time and you don't need all the elements. Then the callback function can return the "stop" flag without completing the list to the end.<br />
<br />
Note: some analogue of user-parameters are <a title="DocWiki: System.Classes.TComponent.Tag" href="https://docwiki.embarcadero.com/Libraries/en/System.Classes.TComponent.Tag"><code>Tag</code></a> and <a href="https://docwiki.embarcadero.com/Libraries/en/Vcl.ComCtrls.TTreeNode.Data" title="DocWiki: Vcl.ComCtrls.TTreeNode .Data"><code>Data</code></a> properties, although their use is not always ideologically correct (correct: create a derived class).<br />
<br />
The conclusion follows from the above: if your API needs to make a callback function, then <b>it must have a custom <code>Pointer</code> size parameter that will not be used by your API</b>. For example:<br />
<pre class="brush:delphi">// Incorrect!
type
TNotifyMeProc = procedure; safecall;
IMyDllAPI = interface
// ...
procedure NotifyMe(const ANotifyEvent: TNotifyMeProc); safecall;
end;</pre>
<pre class="brush:delphi">// Correct
type
TNotifyMeProc = procedure(const AUserArg: Pointer); safecall;
IMyDllAPI = interface
// ...
procedure NotifyMe(const ANotifyEvent: TNotifyMeProc; const AUserArg: Pointer = nil); safecall;
end;</pre>
And if you forget to do this, the caller will have to use ugly hacks to get around your bad API design.<br />
<br />
Naturally, instead of a function + parameter, you can just use an interface:<br />
<pre class="brush:delphi">// Correct
type
INotifyMe = interface
['{07FA30E4-FE9B-4ED2-8692-1E5CFEE4CF3F}']
procedure Notify; safecall;
end;
IMyDllAPI = interface
// ...
procedure NotifyMe(const ANotifyEvent: INotifyMe); safecall;
end;</pre>
This is preferable, because error handling via <code>safecall</code> in interfaces is simpler, and the interface can contain as many parameters as you like, and it is even more convenient to integrate with objects (form). For example:<br />
<pre class="brush:delphi">type
TForm1 = class(TForm, INotifyMe)
// ...
procedure Notify; safecall;
private
FAPI: IMyDllAPI;
public
function SafeCallException(ExceptObject: TObject; ExceptAddr: Pointer): HResult; override;
end;
// ...
procedure TForm1.FormCreate(Sender: TObject);
begin
// ... load DLL, get API
FAPI.NotifyMe(Self); // Ask DLL to call us on some event
end;
procedure TForm1.Notify;
begin
ShowMessage('Something just happenned');
// Our form is available here (in the callback),
// so we can just use it, no need to pass it manually
end;</pre>
<br />
<br />
<a name="t9"></a><h1>Other Rules</h1>
<ol>
<li>If you are not only developing but also using a DLL, then load the DLL correctly. I won't go into details here, as it deserves its own article, but you can take a look at the <code>LoadDLL</code> function from the sample code below: <b>do NOT use <code>LoadLibrary</code></b>!;</li>
<li>If for some reason you don't use <code>safecall</code> then don't return complex types via <code>Result</code>, make it an out parameter. The problem is that Delphi and MS Visual C++ disagree on how to interpret the result returned by reference by a stdcall function: as <code>var</code> or as <code>out</code>. Accordingly, for <code>safecall</code> there is no such problem, since <code>Result</code> for it is always <code>Integer</code> (<code>HRESULT</code>) - a simple type, for which <code>var</code> and <code>out</code> are equivalent;</li>
<li>All APIs must have a unique IID/GUID (non-API interfaces (not mentioned in headers) <i>might</i> not have a GUID, although I would recommend always specifying an IID). You can create a GUID to use as an IID (Interface ID) by pressing <code>Ctrl + Shift + G</code> in the Delphi code editor - this combination will insert an expression like <code>['{C5DBE4DC-B4D7-475B-9509- E43193796633}']</code> (of course, each time with a unique GUID) directly below the cursor in the editor;</li>
<li>Once you have published some type (interface), i.e. released your DLL with this interface - you shouldn't change it. If you need to expand or change it, you introduce a new interface (a new version of the interface), but do not change the old one<br />
<pre class="brush:delphi">// Was:
type
IMyDLL = interface
['{C5DBE4DC-B4D7-475B-9509-E43193796633}']
procedure InitDLL(AOptional: IUnknown = nil); safecall;
procedure DoneDLL; safecall;
function GetDynData(const AFlags: DWORD): IData; safecall;
end;
// You can't do that after publishing a production build:
type
IMyDLL = interface
['{C5DBE4DC-B4D7-475B-9509-E43193796633}']
procedure InitDLL(AOptional: IUnknown = nil); safecall;
procedure DoneDLL; safecall;
procedure DoSomething; safecall; // was added
function GetDynData(const AFlags: DWORD): IData; safecall;
end;
// However, you can do that:
type
IMyDLLv1 = interface
['{C5DBE4DC-B4D7-475B-9509-E43193796633}']
procedure InitDLL(AOptional: IUnknown = nil); safecall;
procedure DoneDLL; safecall;
function GetDynData(const AFlags: DWORD): IData; safecall;
end;
IMyDLLv2 = interface(IMyDLLv1)
['{69E77989-64DC-4177-975C-487818598C70}']
procedure DoSomething; safecall; // added
end;</pre></li>
<li>If a function or method returns an interface, then don't do this:
<pre class="brush:delphi">// Incorrect!
function GetSomething: ISomething; safecall;
// ...
var
Something: ISomething;
begin
Something := GetSomething;</pre>
Surely, it is a convenient solution in the beginning: you can call functions "as usual" and even chain them into chains like <code>Control</code>.<code>GetPicture</code>.<code>GetImage</code>.<code>GetColorInfo</code>.<code>GetBackgroundColor</code>. However, this state of affairs will exist only in the very first version of the system. As soon as you start developing the system, you will start to have new interfaces. In the not so distant future, you will have a bunch of advanced interfaces, and the basic interfaces that were in the program initially, at the time of its birth, will implement only trivially uninteresting functions. As a result, very often the calling code will need new interfaces, not the original ones. What does it mean? This means that the code needs to call the original function, get the original interface, then ask it for a new one (via <code>Supports</code>/<code>QueryInterface</code>) and only then use the new interface. It turns out not so convenient, even rather inconvenient: we have a triple call (original + conversion + required). The best solution is for the calling code to tell the called function which interface it is interested in: the new one or the old one:<br />
<pre class="brush:delphi">// Correct
procedure GetSomething(const AIID: TGUID; out Intf); safecall;
// ...
var
Something: ISomething;
begin
GetSomething(ISomething, Something);</pre></li>
<li>If an object implements an interface, then your code should not contain variables of this class. I.e.:<br />
<pre class="brush:delphi">type
TSomeObject = class(TSomeOtherClass, ISomeInterface)
// ...
end;
var
Obj: TSomeObject; // - incorrect!
Obj: ISomeInterface; // - correct
begin
Obj := TSomeObject.Create;
// ...</pre></li>
<li>If you're implementing an interface extension by inheritance, don't forget to explicitly list all of its ancestors in the implementing object. For example:<br />
<pre class="brush:delphi">type
ISomeInterfaceV1 = interface
['{A80A78ED-5836-49C4-B6C2-11F531103FE7}']
procedure A;
end;
ISomeInterfaceV2 = interface(ISomeInterfaceV1) // ISomeInterfaceV2 is inherited from ISomeInterfaceV1
['{EBDD52A1-489B-4564-998E-09FCCF923F48}']
procedure B;
end;
// Incorrect!
TObj = class(TInterfacedObject, ISomeInterfaceV2) // ISomeInterfaceV2 is mentioned, but not ISomeInterfaceV1
protected
procedure A;
procedure B;
end;
var
SI1: ISomeInterfaceV1;
SI2: ISomeInterfaceV2;
begin
Supports(SI2, ISomeInterfaceV1, SI1);
Assert(Assigned(SI1)); // will fire, since SI1 = nil (Supports returned False)
end;</pre>
A correct way would be:<br />
<pre class="brush:delphi"> // Correct
TObj = class(TInterfacedObject, ISomeInterfaceV1, ISomeInterfaceV2)
// ...</pre></li>
<li>It is not necessary to make the implementation of interface methods virtual:<br />
<pre class="brush:delphi">type
ISomeInterfaceV1 = interface
['{C25F72B0-0BC9-470D-8F43-6F331473C83C}']
procedure A;
end;
TObj = class(TInterfacedObject, ISomeInterfaceV1)
protected
// Incorrect
procedure A; virtual;
end;</pre>
Do like so instead:<br />
<pre class="brush:delphi"> TObj = class(TInterfacedObject, ISomeInterfaceV1)
protected
// Correct
procedure A;
end;</pre></li>
<li>Do not use the <code>const</code> modifier with interface parameters:<br />
<pre class="brush:delphi">// Incorrect!
procedure DoSomething(const AArg: ISomething); safecall;
// Correct
procedure DoSomething(AArg: ISomething); safecall;</pre></li>
<li><a title="Basic ground rules for programming – function parameters and how they are used" href="https://devblogs.microsoft.com/oldnewthing/20060320-13/?p=31853">Other unspoken rules</a>.</li>
</ol>
<br />
<br />
<a name="t10"></a><h1>Conclusion</h1>
<a href="https://www.eurekalog.com/getfile.php?id=92" title="Download SampleDllAPI.zip">Download sample DLL API here</a>. The archive contains a group of two projects (a DLL and an application that uses it). The DLL implements an sample API with example functions. The SDK folder contains the SDK, which consists of:<br />
<ul>
<li>SDK:
<ul>
<li><code>SampleDLLHeaders.pas</code> header file;</li>
<li>CHM and PDF documentation (and project sources <a title="Help + Manual Website" href="https://www.helpandmanual.com/">Help&Manual</a>);</li>
</ul></li>
<li>As well as the Delphi-specific support file <code>DelphiSupport.pas</code>.</li>
</ul>
Developers in other programming languages can use <code>SampleDLLHeaders.pas</code> and Delphi developers can use <code>SampleDLLHeaders.pas</code> + <code>DelphiSupport.pas</code>.<br />
<br />
Headers are presented only in the form of Delphi code. Translation into other programming languages left as homework.<br />
<br />
The <code>DelphiSupport.pas</code> module can be included both in the DLL and in applications that use it. It contains:<br />
<ul>
<li>Processing <code>safecall</code> (along with fixing <a href="https://quality.embarcadero.com/browse/RSP-24652" title="Quality Portal: [RSP-24652] TObject. SafeCallException might be NOT called in certain cases">RSP-24652</a>);</li>
<li>A base object <code>TBaseObject</code> for implementing interfaces with <code>safecall</code> processing support and debugging checks;</li>
<li>Prebuilt classes: <code>TMalloc</code> allocator and <code>TNotify</code> wrapper;</li>
<li>The function <code>RegisterInitFunc</code> for registering the initialization of modules in the DLL;</li>
<li><code>LoadDLL</code> function for correct loading of the DLL.</li>
</ul>
<br />
The DLL API has sample functions:<br />
<ul>
<li>Returning an array of strings <code>GetData</code>;</li>
<li>Returning dynamic memory <code>GetMemory</code>;</li>
<li>Callback function set by <code>NotifyMe</code>;</li>
<li>Error/Exception Test <code>TryAbort</code>, <code>TryAccessViolation</code>, etc</li>
</ul>
<br />
The calling application shows both "load-use-upload" and "load, use, upload".GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-21509632043764452372023-08-12T16:51:00.003+01:002023-08-12T17:16:03.470+01:00What can you do with the "Assember" tab in bug reports? Is it even useful?We were contacted by a customer that claimed that his application worked fine until he added EurekaLog to it. Specifically, his application starts to raise an <code>EAccessViolation</code> exception with the following message:<br />
<code>Access violation at address 03DB472F in module 'Sample.exe'. Read of address 5653E4CC</code><br />
<a name='more'></a><br />
The customer was kind enough to share the code mentioned in the call stack of the bug report:
<pre class="brush:delphi">function TContosoEventCollection.FindEvent(const AValue: TContosoEvent): Integer;
var
X: Integer;
begin
Result := -1;
if not Assigned(FEvents) then
Exit;
for X := 0 to FEvents.Count - 1 do // - crashes here
begin
if SameMethod(AValue, FEvents[X]) then
begin
Result := X;
Break;
end;
end;
end;</pre>
Where <code>FEvents</code> is the <code>TList</code> field in the <code>TContosoEventCollection</code> class.<br />
<br />
As a reminder: the <code>TList</code> class is implemented in the RTL like this:<br />
<pre class="brush:delphi">type
TList = class(TObject)
private
FList: TPointerList;
FCount: Integer;
// ...
protected
function Get(Index: Integer): Pointer; // returns FList[Index]
// ...
public
property Items[Index: Integer]: Pointer read Get write Put; default;
property Count: Integer read FCount write SetCount;
// ...
end;</pre>
So, the code crashes while trying to read the <code>FCount</code> field of the <code>TList</code> class - which probably means that the <code>TList</code> object is not valid (in other words: trashed).<br />
<br />
Well, the bug report can not be false-positive. So, what happenned? And why there is no access violation in the same application compiled without EurekaLog?<br />
<br />
Quite often you would see message like this for memory errors:<br />
<code>Access violation at address ... in module '...'. Read of address DEADBEEF</code><br />
Where the data address would match or be close to the DEADBEEF value - this is the debug marker that EurekaLog fills in the memory being deleted. In addition to DEADBEEF, you can also encounter other values: $CCCCCCCC, $FEEEFEEE, $FDFDFDFD, $FADEDEAD, $00009999. EurekaLog attempts to reserve at least one of them for debugging purposes and will use the first one for which the reservation succeeds. There may also be values close to them (i.e. debug marker + offset, e.g. DEADBE8F).<br />
<br />
However, in this particular case: the data address is 5653E4CC - which does not match any known debugger marker. So, at the first glance it <i>could</i> be a valid data value.<br />
<br />
Note that 5653E4CC is supposed to be an address of the <code>TList</code>'s <code>FCount</code> field. Now, let's take a look at the "Assember" tab from EL's bug report:
<pre class="brush:asm">; Line=3297 - Offset=23
; ---------------------
03DB472F 8B7808 MOV EDI, [EAX+8] ; <-- EXCEPTION</pre>
Now we can see that the <code>TList</code>'s <code>FCount</code> field has the offset of 8 from the start of the <code>TList</code> object. In other words: it is the second field in the class. Therefore, the address of the <code>TList</code> object itself (in other words: the <code>FEvents</code> field in customer's <code>TContosoEventCollection</code> object) is 5653E4CC - 8 = 5653E4C4.<br />
<br />
Why is this important?<br />
<br />
Well, the 5653E4C4 = 1448338628 and it is <b>NOT</b> divisible by 16. In fact, it is not even disisible by 8. It matters because allocation granularity on Delphi is at least 16 bytes.<br />
<br />
Conclusion: 5653E4C4 can not be address of any <code>TList</code> object in customer's application. Which means that it is a completely trash value. (Otherwise, we might suspect that it could be a value of <i>some</i> <code>TList</code> object - the one that was already deleted.)<br />
<br />
Summary: the <code>FEvents</code> (<code>TList</code>) field in customer's <code>TContosoEventCollection</code> object is trashed.<br />
<br />
Let's dig further. Look at the "Assembler" tab again:<br />
<pre class="brush:asm">; ContosoUnit.TContosoEventCollection.FindEvent (Line=1234 - Offset=0)
; ---------------------------------------------------------
03DB4718 55 PUSH EBP
03DB4719 8BEC MOV EBP, ESP</pre>
This sounds like a start of the <code>TContosoEventCollection.FindEvent</code> method to me. And what do we know about Delphi methods? Well, they pass <code>Self</code> as the implicit first argument. And what do we know about arguments passing in x86? Well, the first argument is stored in the <code>EAX</code> CPU register. It is further confirmed by the same "Assember" tab:<br />
<pre class="brush:asm">03DB471F 8BF0 MOV ESI, EAX
; if not Assigned(FEvents) then
03DB4728 8B4604 MOV EAX, [ESI+4]
03DB472B 85C0 TEST EAX, EAX</pre>
We can clearly see as <code>Self</code> is being copied from <code>EAX</code> into <code>ESI</code>, and then <code>ESI</code> is being used to read the <code>FEvents</code> field (which, as we remember, is trashed) - which has offset of 4 (e.g. the very first field in the class - also confirmed by customer's source code).<br />
<br />
So, when crash occurs, the <code>Self</code> must be stored in the <code>ESI</code> CPU register. Let's take a look at it: 00451C10.<br />
<br />
Do you see anything wrong with it?<br />
<br />
Well, while the 00451C10 value <b>is</b> divisible by 16, but customer's EXE loads at the default 00400000 address - which is awfully close to the 00451C10. In fact, customer's app has crashed at the 03DB472F address (see the original exception's message) - which is well inside reach from the 00400000, given size of customer's exe being more than 100'000'000 bytes (as inspected from the "Modules" tab). 100'000'000 = 05F5E100, so 00400000 + 05F5E100 = 0635E100 - way more than the 03DB472F. Therefore, any valid data/objects must have addresses only above 0635E100 (or below 00400000).<br />
<br />
In other words: 00451C10 is a code address. It can not possibly be any data/<code>TObject</code> address.<br />
<br />
Conclusion: customer's <code>TContosoEventCollection</code> object is trashed too, so no wonder we read trash data from it, so its <code>FEvents</code>/<code>TList</code> field also becomes trash. Customer was unlucky in the sense that the invalid value of the <code>TContosoEventCollection</code> object was, in fact, readable, so CPU could load the (trashed) <code>FEvents</code> field from it and only crashed later. If customer was lucky - he would crash immediately on access to <code>TContosoEventCollection</code>'s <code>FEvents</code> field.<br />
<br />
Summary: customer should look further up stack to the caller code: it is calling the <code>FindEvent</code> method on the invalid object value. From where does it reads/takes the <code>TContosoEventCollection</code> object? Whatever it was - it is trashed now.<br />
<br />
The customer has reported that he was able to find and fix the issue armed with this information.<br />
<br />
As for the question why there was no access violation in the same application compiled without EurekaLog - well, applications without EurekaLog have absolutely no memory checks in them! So <a href="https://blog.eurekalog.com/2023/04/librariescomponentsarenottested.html" title="Many libraries/components are not tested for memory bugs">most memory-related errors go unnoticed</a>.<br />
<br />
P.S. <a href="https://blog.eurekalog.com/search/label/Stories" title="EurekaLog Blog: Stories">Read more stories like this one</a> or <a href="https://www.eurekalog.com/casestudies.php" title="Case Studies and User Reviews">read feedback from our customers</a>.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-26762435677420082362023-07-26T17:33:00.009+01:002023-08-09T13:13:00.045+01:00Does EurekaLog support CLang/LLVM/64-bit?<h1>Short Answer</h1>
Yes, EurekaLog supports CLang and LLVM in both 32 and 64 bits - with some limitations.<br />
<a name='more'></a><br />
<h1>Long Answer</h1>
<h3>History</h3>
The 64-bit Delphi is build on the classic Borland toolchain. EurekaLog 7 supported 64-bit Delphi since the very begining: the original release of EurekaLog 7.0.0.0 in 2012. Since many of our clients were using the new 64-bit platform, we were also able to improve its support over time. It was feature-complete in somewhere near 2014. Currently it does not have any limitations.<br />
<br />
On the other hand, the 64-bit C++ Builder is build on entirely different toolchain: LLVM. Classic toolchain is not used for 64-bit C++ Builder. The CLang is LLVM's frontend for C++. Initially EurekaLog 7 did not have support for LLVM.<br />
<br />
While 32-bit C++ Builder was also originally build on the classic Borland toolchain, Embarcadero has released RAD Studio 10 Seattle in 2015 with an option to use the LLVM-based toolchain for 32-bit C++ Builder. In other words, starting with RAD Studio 10 Seattle (2015+):<br />
<ul>
<li>32-bit Delphi: classic only</li>
<li>64-bit Delphi: classic only</li>
<li>32-bit C++ Builder: classic OR LLVM - your choice</li>
<li>64-bit C++ Builder: LLVM only</li>
</ul>
<br />
Since now CLang was accessible in 32-bit, we also started implementing support for LLVM and CLang. However, not many of our clients use C++ Builder. And very few clients immediately switched to CLang/LLVM. Therefore, we also couldn't polish our CLang/LLVM support as quickly as Delphi's 64-bit support. However, a lot of time has passed and we can say with confidence that many of our clients use EurekaLog for CLang/LLVM today.<br />
<br />
Support for 64-bit C++ Builder is a relatively recent addition. It was enabled in 2020. EurekaLog doesn't track usage statistics, so we don't know exactly how many of our clients are using 64-bit C++ Builder. But we can judge this indirectly: by the number of questions to our technical support. And judging by it: almost none of our clients use 64-bit C++ Builder. This means we don't get a lot of feedback and improvements to our 64-bit C++ Builder support are VERY slow.<br />
<br />
In summary: while EurekaLog does support CLang, LLVM and 64-bit, you must understand that some aspects had more time to polish and evolve. For these historical reasons, the levels of maturnity (in order from high to low) are:
<ol>
<li>32-bit Delphi</li>
<li>32-bit C++ Builder on classic toolchain</li>
<li>64-bit Delphi</li>
<li>32-bit C++ Builder on LLVM toolchain (CLang)</li>
<li>64-bit C++ Builder</li>
</ol>
<br />
<h3>Technical details and limitations</h3>
As mentioned above, 32-bit and 64-bit Delphi support and 32-bit C++ Builder on classic toolchain are feature-complete and have no limitations.<br />
<br />
Unfortunately, using LLVM (CLang) for 32 or 64-bits means that some exception info is missing. LLVM (or its implementation in C++ Builder) is very unpolished for exception tracing. Specifically, it does not set an exception address, it does not set a valid stack pointer and a frame - all those necessary and important values are just zeros. We really got into the wilds of workarounds and hacks, only to extract the address of the exception. Everything really hangs on straws and assumptions.<br />
<br />
Classic C++ Builder is much better in this aspect. We are improving diagnostic as much as possible, so newer versions of EurekaLog should provide better info. But there may be something that is impossible to implement from our side. We highly recommend that you don't use LLVM on 32-bit, use classic compiler. Naturally, you are limited to LLVM only for 64-bit C++ Builder as there is no option to use classic compiler.<br />
<br />
Therefore, stack frames based methods will not work if you are using LLVM/CLang compiler (in other words: "Use classic compiler" option is disabled). Select any RAW method for LLVM/CLang.<br />
<br />
One other point to consider is the <a href="https://www.eurekalog.com/help/eurekalog/configuring_project_itself.php" title="Configuring project for EurekaLog">debug information</a>. Delphi, C++ Builder and RAD Studio has an option to generate a text .map file, which contains debug information about the compiled module - such as unit, class, function names and line numbers. <a href="https://www.eurekalog.com/help/eurekalog/eurekalog_basics.php" title="EurekaLog's Basics">Such .map file is used by EurekaLog to compose debug information in EurekaLog's own format</a> - designed for maximum access speed or minimum space. The issue is that .map files produced by the C++ Builder do not have line numbers - just names of units, classes, and functions:<br />
<ol>
<li>EurekaLog is also able to read debug information in Turbo Debugger (TD32, TDS) format. If it is enabled for C++ Builder - EurekaLog will use it to extract line numbers. So, when using 32-bit C++ Builder on the classic toolchain - EurekaLog will be able to show line numbers in bug reports.</li>
<li>If you would use LLVM/CLang on Win32 (e.g. "Use Classic compiler" option is disabled) or 64-bit C++ Builder (which is LLVM/CLang-only) - then no TD32/TDS debug information will be available, as LLVM/clang generates DWARF debug information, not TD32/TDS. Currently EurekaLog is unable to read DWARF debug information, so there will be no line numbers in bug reports.</li>
</ol>
<br />
You can help us to fix this issue by voting for this entry: <a title="Quality Portal: RSP-27359" href="https://quality.embarcadero.com/browse/RSP-27359">[RSP-27359] No line numbers in map files for LLVM-based compilers</a>.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-48777529961562565282023-04-03T21:43:00.003+01:002023-04-04T09:24:47.365+01:00Many libraries/components are not tested for memory bugsWe were contacted by a customer, who claimed that EurekaLog causes Access Violation in a simple sample demo application.<br />
<br />
Specifically, the application runs fine when compiled without EurekaLog and produces the expected results. But application crashes with "Access violation at address 00410759 in module 'DemoApp.exe'. Read of address 83EC8B59" when compiled with EurekaLog.<br />
<a name='more'></a><br />
Unfortunately, the customer did not report any additional details, such as: EurekaLog version, IDE version, OS version, bug report file nor call stack, etc. However, we did know that the access violation exception happenned inside the <code>System._IntfClear</code> function.<br />
<br />
Well, it doesn't take a genius to figure out that there is most likely a memory bug in the sample app's code. And it is probably something related to mixing manual/automatic management of object's lifecycle.<br />
<br />
We have installed a demo version of the components in question and compiled a sample application with EurekaLog. Running the application and simply exiting it trigered the following error:<br />
<br />
"Application made attempt to call method of already deleted object: $0430B610 OBJECT [TContosoDoc] 340 bytes"<br />
<br />
while the call stack from the bug report looked like this:<br />
<br />
<code>System._IntfClear<br />
System._FinalizeRecord<br />
System.TObject.CleanupInstance<br />
System.TObject.FreeInstance<br />
System._ClassDestroy<br />
Contoso.VCL.TContosoPropertiesForm.Destroy 2893[2]<br />
System.TObject.Free<br />
System.Classes.TComponent.DestroyComponents<br />
Vcl.Forms.DoneApplication<br />
System.SysUtils.DoExitProc<br />
System._Halt0<br /></code>
<br />
Well, the error message is different, but the crash location is the same. Why is that?<br />
<br />
There could be few reasons for that:<br />
<ol>
<li>The customer did not report when and how the error occurs. Perhaps, he was seeing a different error in a different place.</li>
<li>The client could (correctly) assume that it was a memory bug somewhere, so he tried to "fix" the bug by disabling some of the EurekaLog's memory check options. Therefore, the client application's configuration could differ from defaults. And we were checking using the default settings.</li>
<li>The code that detects/shows the "call method of already deleted object" message relies on the fact that the released memory remains untouched. However, if some code allocates memory over this disposed memory, the check could not function, and you get a simple access violation instead. So, depending on how memory is allocated/disposed, the behaviour can change.</li>
</ol>
<br />
Anyway, the EurekaLog was able to show a second call stack for the same object: specifically, the second call stack shows where the object was originally destroyed:<br />
<br />
<code>ContosoDoc.TContosoCustomDoc.Destroy<br />
System.Classes.TComponent.DestroyComponents<br />
Vcl.Forms.DoneApplication<br />
System.SysUtils.DoExitProc<br />
System._Halt0<br /></code>
<br />
Just looking at these two call stack you can see the problem already:
<ol>
<li>The second call stack (actual deletion) mentions that the object in question was deleted "manually" by calling its destructor when components are cleaned up on app's shutdown.</li>
<li>The first call stack (access to already deleted object) mentions that the same object was also tried to be deleted automagically via an interface reference.</li>
</ol>
Thankfully, we don't need the source code for the component/library (which we don't have, because we are using demo/trial) to confirm that. The line number from the first call stack leads us directly to the problem:<br />
<pre class="brush:delphi">constructor TContosoPropertiesForm.Create(AOwner: TComponent);
begin
inherited;
FDoc := TContosoDoc.Create(Self);
end;
destructor TContosoPropertiesForm.Destroy;
begin
inherited;
end;</pre>
where the <code>FDoc</code> field is declared as:<br />
<pre class="brush:delphi">private
FDoc: IContosoDoc;</pre>
Do you see the problem?<br />
<br />
The <code>TContosoDoc</code> will be deleted by the <code>TContosoPropertiesForm</code>, because the <code>TContosoPropertiesForm</code> (<code>Self</code>) was passed as an owner to the <code>TContosoDoc</code>. So, when the <code>TContosoPropertiesForm</code> deletes itself - it also deletes all owned sub-components, including the <code>TContosoDoc</code>.<br />
<br />
But the reference to the <code>TContosoDoc</code> was also saved into the <code>FDoc</code> field. That should not be a problem if the field has the <code><b>T</b>ContosoDoc</code> type. But it has the <code><b>I</b>ContosoDoc</code> type. In other words, it is an interface! When interface goes out of scope, it dereferences, and the object is deleted when the reference count reaches zero.<br />
<br />
You may know that the components (descendants from <code>TComponent</code>) override the automatic inteface management by saying "there is no reference counter". In other words, increasing and descreasing interface counter do absolutely nothing.<br />
<br />
If so - why there is the crash then? The reason is that even the simple "there is no reference counter" behavior requires <b>virtual</b> method calls! Indeed, remember that <code>IInterface</code>/<code>IUnknown</code> is declared as:<br />
<pre class="brush:delphi">type
IInterface = interface
['{00000000-0000-0000-C000-000000000046}']
function QueryInterface(const IID: TGUID; out Obj): HResult; stdcall;
function _AddRef: Integer; stdcall;
function _Release: Integer; stdcall;
end;
IUnknown = IInterface;</pre>
In other words, any interface in Delphi <b>must</b> implement the <code>_AddRef</code> and <code>_Release</code> methods, because all interfaces in Delphi descent from <code>IInterface</code>.<br />
<br />
Another piece of the puzzle: the "there is no reference counter" behaviour is not implemented like "do not call the <code>_AddRef</code>/<code>_Release</code> methods". Instead, this behaviour is implemented like "the <code>_AddRef</code>/<code>_Release</code> methods do nothing". So, the <code>_AddRef</code>/<code>_Release</code> methods must be called.<br />
<br />
If so - they must be implemented (as emtpy methods). And how do you implement interface methods? By using virtual methods:<br />
<pre class="brush:delphi">type
TInterfacedObject = class(TObject, IInterface)
protected
FRefCount: Integer;
function QueryInterface(const IID: TGUID; out Obj): HResult; stdcall;
function _AddRef: Integer; stdcall;
function _Release: Integer; stdcall;
end;</pre>
Yes, the <code>_AddRef</code>/<code>_Release</code> methods are non-virtual methods for <b>the object (class)</b>. However, remember that interface is basically an abstract class, which means all of its methods are pure virtual. Which means that the mentioned methods will be virtual methods for <b>the interface</b> once it is implemented.<br />
<br />
And how do you call a virtual method? Well, you have to look it up inside object's (interface's) virtual method table. But if the object/interface was already released then its virtual method table won't be accessible anymore. That is where the bug comes from. The code is not actually trying to delete already deleted object, but it is trying to say "interface goes out of scope, please decrease reference counter". Normally this would result in the "do nothing" behaviour, but in our case the "do nothing" behaviour could not be located, since its implementing object is already gone.<br />
<br />
So, why it was not a problem without EurekaLog on board?<br />
<br />
It's simple: deleting object means marking its memory as "empty". The memory itself is not gone. And its content stays the same. Therefore, any futher calls to <code>_AddRef</code>/<code>_Release</code> methods will be successful, since virtual method table still could be located.<br />
<br />
Conclusion: it is a bug in library/component's demo code, which must be fixed. Simplest way is a workaround: set the <code>FDoc</code> field to <code>nil</code> as the first action in the <code>TContosoPropertiesForm</code>'s destructor. One correct way to fix it is to change field's type to object (class), so interfaces will be created/disposed only when used. Another way is to remove ownership and implement the reference counting, so object's lifetime will be managed by interface field only.<br />
<br />
Moral of the story: use either interfaces or objects, do not mix! E.g. if you use interfaces - do not store references to implementing objects. If you use objects - do not store interface references.<br />
<br />
As you can imagine, many libraries and components come with memory-related bugs, because there is no build-in tools in Delphi to diagnose such issues. You need a 3rd party tool: debugging memory manager. Not every library/component vendor will go extra length to use 3rd party tool to test his code. This is true even for Delphi itself, as both VCL and FMX has similar memory bugs which usually stays hidden. For example: <a href="https://quality.embarcadero.com/browse/RSP-38694" title="RSP-38694: Using 'with' in VCL causes 'use after free' bugs">RSP-38694</a>, <a href="https://quality.embarcadero.com/browse/RSP-30403" title="RSP-30403: Leak in TParallel.For / EAggregateException">RSP-30403</a>, <a href="https://quality.embarcadero.com/browse/RSP-28294" title="RSP-28294: Regression for RSP-10308: accessing already deleted object">RSP-28294</a>, <a href="https://quality.embarcadero.com/browse/RSP-10308" title="RSP-10308: TWebBrowser has a memory leak">RSP-10308</a>, ...<br />
<br />
So, what if you can't fix the 3rd party code? Well, you can hide the bug by <a href="https://www.eurekalog.com/help/eurekalog/memory_leaks_page.php" title="EurekaLog Memory Options">disabling the memory checks in EurekaLog</a>. We recommend that you keep the "Enable extended memory manager" option enabled and disable all other sub-options. Don't forget to set the "When memory is released" option to "Do nothing". Please note that by doing so - you are hiding the bug, you are not actually fixing it!<br />
<br />
P.S. It might be counter-intuitive to some, but if you want to fix a memory bug - you need to <b>enable</b> the "Catch memory leaks" option (make sure the "Active only when running under debugger" option is off if you are running the app outside of the debugger). Enabling memory leaks checks allows EurekaLog to allocate additional memory blocks with information about allocated memory. In these additional memory blocks, EurekaLog can store, among other things, additional call stacks and information about the memory's data type. All this additional information can help EurekaLog produce more accurate diagnostic information if a problem was found.<br />
<br />
P.P.S. <a href="https://blog.eurekalog.com/search/label/Stories" title="EurekaLog Blog: Stories">Read more stories like this one</a> or <a href="https://www.eurekalog.com/casestudies.php" title="Case Studies and User Reviews">read feedback from our customers</a>.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-64850408564054176482022-12-01T09:57:00.003+00:002022-12-01T09:57:34.791+00:00EurekaLog 7.11 is outWe are pleased to announce the availability of the new 7.11 version.<br />
<a name='more'></a><br />
<div style="text-align: center;">
<a href="https://www.eurekalog.com/" target="_blank" title="EurekaLog Logo"><img alt="EurekaLog Logo" src="https://www.eurekalog.com/images/logo.png" height="68" width="295" /></a></div>
<blockquote>
EurekaLog is a tool for Delphi and C++Builder that gives your application the power to catch every exception and memory/resource leak, generating a detailed log of the call stack (with unit, procedure and line number), optionally sending you a copy of each log entry via email or the Web via the most used Web Bug-Tracking tools (as Mantis, BugZilla, FogBugz, JIRA, YouTrack, Redmine, Exceptionless, GitLab, and GitHub).</blockquote>
EurekaLog represents the most advanced exception and memory leaks logger technology available for Delphi/C++Builder developers.<br />
<br />
To learn more about EurekaLog, please visit our website at:<br />
<a href="https://www.eurekalog.com/" target="_blank" title="https://www.eurekalog.com">https://www.eurekalog.com</a><br />
<br />
To download a demo, please visit:<br />
<a href="https://www.eurekalog.com/downloads.php" target="_blank" title="https://www.eurekalog.com/downloads.php">https://www.eurekalog.com/downloads.php</a><br />
<br />
If you would like to purchase a new license, please visit:<br />
<a href="https://www.eurekalog.com/buy.php" target="_blank" title="https://www.eurekalog.com/buy.php">https://www.eurekalog.com/buy.php</a><br />
<br />
<strong>Changes in 7.11 build:</strong><br />
<ol>
<li>Added: Support for RAD Studio 11.2 (update 2 for 11 Alexandria)</li>
<li>Added: EurekaLog can now be packaged into a custom BPL package. Options from BPL package will be used to initialize EurekaLog. EurekaLogCore continues to be a default RTL package, which will use options from main executable</li>
<li>Added: [C++ Builder] Support for CLang/LLVM objects for memory features. <b>Do not forget to update your <code>EMemLeaksBCB.cpp</code> files from EurekaLog installation folder</b></li>
<li>Added: [C++ Builder] Support for memory features for 64-bits</li>
<li>Added: [C++ Builder] New <code>_TDSComplete</code> option can now be used to turn off TDS processing for faster post-processing of C++ Builder projects (no line numbers will be added), good for quick testing, not recommended for production</li>
<li>Added: Stay On Top checkbox for MessageBox dialog</li>
<li>Added: New <code>.LastSystemErrorCode</code> property for exception info</li>
<li>Added: [Viewer] EurekaLog report version is now displayed as the first value under "General" tab</li>
<li>Added: A warning about saving your project before editing EurekaLog's options</li>
<li>Added: EurekaLog options will now have an unique GUID set</li>
<li>Added: Ability to add debug information/options to an external file (.edbg) instead of injecting inside executable. Can be used for C++ Builder when debugger is unable to recognize the modified .exe. Additionally, there is a hidden option <code>DoNotTouchExe</code> (for testing or packers/digital signatures)</li>
<li>Added: Ability to side-load external debug info for main module (for example, use EurekaLog options, but use JCL debug info)</li>
<li>Added: Made <code>SMTPClientAdditionalHeaders</code> to be additional argument for the <code>EurekaLogSendEmail</code> functions</li>
<li>Fixed: Custom/Help button will not be visible if no event handler is assigned</li>
<li>Fixed: [64-bit C++ Builder] IDE debugger crash when debugging EurekaLog-enabled executables</li>
<li>Fixed: [C++ Builder] Various memory diagnostic improvements</li>
<li>Fixed: [C++ Builder] Proper unit handling in IDE when switching target platform</li>
<li>Fixed: Added workaround for <a title="My application starts throwing 12175/ERROR_WINHTTP_SECURE_FAILURE error after adding EurekaLog" href="https://support.eurekalog.com/index.php?/Knowledgebase/Article/View/93/10/7x-my-application-starts-throwing-12175error_winhttp_secure_failure-error-after-adding-eurekalog">this issue</a></li>
<li>Fixed: Added support latest changes in JIRA API</li>
<li>Fixed: Taking screenshots from non-DPI awared applications</li>
<li>Fixed: [x64] Rare crash when following error line from report in IDE</li>
<li>Fixed: [Regression] MS Debug Info provider may fail to obtain information</li>
<li>Fixed: [Regression] Showing EurekaLog dialog may hung when parent window is not responding</li>
<li>Fixed: [Regression] Rare deadlock on startup when using MS Debug Info provider</li>
<li>Fixed: [Regression] ECC32 may fail to compile project when no .dproj/.cbproj file is specified</li>
<li>Fixed: Improved support for non-default output file extensions</li>
<li>Fixed: Freeze detection restart controls improvements</li>
<li>Fixed: EurekaLog style dialog will property set a default button</li>
<li>Fixed: Sometimes EurekaLog dialog could appear behind app's window</li>
<li>Fixed: Rare range-check error in exception dialogs</li>
<li>Fixed: Rare crash on startup of RAD Studio 11 Alexandria (bug in the workaround for the RSP-36484)</li>
<li>Fixed: [Regression] SMTP send speed improvements</li>
<li>Fixed: [C++ Builder] Incorrect unit initialization order could cause EurekaLog to ignore exceptions</li>
<li>Fixed: [C++ Builder] Installation of library paths for 64-bit</li>
<li>Fixed: [C++ Builder] Crash on startup in C++ Builder 2010 only</li>
<li>Fixed: Minor internal logging improvements</li>
<li>Fixed: Rare issues with some SMTP servers</li>
<li>Fixed: "Include child classes" option did not work correctly in some cases (mostly on older IDEs)</li>
<li>Fixed: <a href="https://www.eurekalog.com/help/eurekalog/how_to_register_event_handler_for_leaks.php" title="How to register event handler for leaks">OnExceptionNotify now works for leaks too</a></li>
<li>Fixed: Various minor improvements</li>
<li>Removed: Hook for <code>CreateFileA</code> function as no longer used by EurekaLog's run-time code. Now this hook is used only for design-time</li>
<li>Removed: The "Delete service files after compilation" option. You can add the "<code>del "%_IDEDst%*.map";del "%_IDEDst%*.drc";del "%_IDEDst%*.dcu"</code>" command (without external quotes) to <a href="https://www.eurekalog.com/help/eurekalog/build_events_page.php" title="Build Events">EurekaLog's post-build event</a>.</li>
<li>Changed: FastMM support units will no longer include debug information providers. You are supposed to manually add/enable providers that you want/need/use.</li>
<li>Changed: [IDE] New code for EurekaLog version update checks</li>
</ol>GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-62074376141868215602022-11-21T05:00:00.001+00:002022-11-21T05:00:00.190+00:00Black Friday and Cyber Monday SaleWe are pleased to announce Black Friday and Cyber Monday <b>30% discount</b> on any of our EurekaLog products using the coupon code found below.<br />
<br />
The sale starts on Black Friday (November, 25) and ends at the end of Cyber Monday (November, 28).<br />
<br />
Enter this code when paying for the item on our web site:<br />
<br />
<b>BFCM2022</b><br />
<br />
<hr />
<br />
Existing customers with valid or expired licenses can log in and purchase upgrades, new licenses and extensions here:<br />
<br />
<a href="https://www.eurekalog.com/login.php" title="Log in to customer control panel">https://www.eurekalog.com/login.php</a><br />
<br />
Use the login credentials we sent you at purchase time.<br />
<br />
<hr />
<br />
New customers (without existing licenses) can use the discount code here:<br />
<br />
<a href="https://www.eurekalog.com/buy.php" title="Purchase EurekaLog">https://www.eurekalog.com/buy.php</a>GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-35712932672132815292022-10-05T10:34:00.001+01:002022-10-05T10:38:41.182+01:00EurekaLog helps not only you, but also developers of libraries that you are usingWe were contacted by a customer that claimed that his application worked fine until he added EurekaLog to it. Specifically, his application starts to raise an <code>EAccessViolation</code> exception in his event handler's code.<br />
<a name='more'></a><br />
A quick look indicated that the <code>EAccessViolation</code> exception was thrown with the following message: "<code>Access violation at 0x0108164d: read of address 0xdeadc30f</code>" and the following call stack:
<pre>ContosoInplaceContainer.TContosoCustomViewInfoItem.Destroy
ContosoInplaceContainer.TContosoEditCellViewInfo.Destroy
System.TObject.Free
ContosoClasses.TContosoObjectList.FreeItem
ContosoClasses.TContosoObjectList.Clear
ContosoGrid.TContosoCustomRowViewInfo.ClearValuesInfo
ContosoGrid.TContosoCustomMultiEditorRow.EditorsChanged
ContosoGrid.TContosoEditorPropertiesCollection.Update
System.Classes.TCollection.Changed
System.Classes.TCollection.RemoveItem
System.Classes.TCollectionItem.SetCollection
System.Classes.TCollectionItem.Release
System.Classes.TCollectionItem.Destroy
ContosoGrid.TContosoCustomEditorRowProperties.Destroy
System.TObject.Free
System.Classes.TCollection.Delete
Unit2.TForm2.DeleteEditor
Unit2.TForm2.PropertiesEditValueChanged
ContosoEdit.TContosoCustomEditingController.EditValueChanged
ContosoInplaceContainer.TContosoEditingController.EditValueChanged
...</pre>
The code in question is:
<pre class="brush:delphi">destructor TContosoCustomViewInfoItem.Destroy;
begin
// Crashes below:
if (Control <> nil) and (Control.Controller.HotTrackController.HintElement = Self) then
Control.Controller.HotTrackController.CancelHint;
inherited Destroy;
end;</pre>
The "Contoso" refers to Contoso Ltd. - which is a <a href="https://devblogs.microsoft.com/oldnewthing/20061013-05/?p=29393" title="Why do I see the same fake names in Microsoft samples over and over?">fictional company used by Microsoft as an example company</a>. Here it masks the real vendor of a certain well-known 3rd party library, as the point of this story is to show how EurekaLog can help you find bugs, not to ridicule any particular developer/vendor.<br />
<br />
Unlike most <a href="https://blog.eurekalog.com/search/label/Stories" title="EurekaLog: Stories">other stories</a> this one was extremely easy to resolve. Mostly because we had a reliable reproducible example.<br />
<br />
First of all, take a closer look at the exception message: "...read of address 0xdeadc30f". Notice that the code is trying to access the DEADC30F address, which is close to the DEADBEEF address. The DEADBEEF is a special debugging marker (see the <a href="https://www.eurekalog.com/help/eurekalog/memory_leaks_page.php" title="EurekaLog Memory Options">"When memory is released" option</a>), indicating already released memory. In other words, <b>this code is trying to access already deleted object</b>.<br />
<br />
The above means that our customer (or 3rd pary code, e.g. Contoso) have "use after free" bug somewhere. This is 100% reliable information, which <b>can NOT be false-positive</b>. In other words, if customer is sure that his code is correct - then he has found a bug in the Contoso library (congratulations!). And visa versa: if Contoso library code is correct, then there is a bug in customer's code on how he uses the Contoso library.<br />
<br />
Armed with this information all there is left to do is to simply walk through customer's code, paying attention to all delete/free operations. Here is how it goes:<br />
<ol>
<li>The <code>TContosoCustomEditingController.EditValueChanged</code> fires an event handler, which is set to customer's code (<code>PropertiesEditValueChanged</code>), which calls: <code>Row1.Properties.Editors.Delete(1);</code>.</li>
<li>The <code>Delete</code> is a method of RTL's <code>TCollection</code>, which removes item from the collection and then deletes the item:<br />
<pre class="brush:delphi">procedure TCollection.Delete(Index: Integer);
begin
Notify(TCollectionItem(FItems[Index]), cnDeleting);
TCollectionItem(FItems[Index]).DisposeOf; // here
end;
destructor TContosoCustomEditorRowProperties.Destroy;
begin
FreeAndNil(FEditContainer); // IMPORTANT
inherited Destroy; // here
end;</pre>
Notice that the <code>TContosoCustomEditorRowProperties.Destroy</code> is deleting the <code>FEditContainer</code> field. However, there is a reference to that object in other place - as we will see below.</li>
<li>Now <code>TCollectionItem</code>'s destructor will release itself from the owner (collection):<br />
<pre class="brush:delphi">destructor TCollectionItem.Destroy;
begin
if FCollection <> nil then
Release; // here
inherited Destroy;
end;
procedure TCollectionItem.Release;
begin
SetCollection(nil); // here
end;
procedure TCollectionItem.SetCollection(Value: TCollection);
begin
if FCollection <> Value then
begin
if FCollection <> nil then FCollection.RemoveItem(Self); // here
if Value <> nil then Value.InsertItem(Self);
end;
end;
procedure TCollection.RemoveItem(Item: TCollectionItem);
begin
Notify(Item, cnExtracting);
if Item = FItems.Last then
FItems.Delete(FItems.Count - 1)
else
FItems.Remove(Item);
Item.FCollection := nil;
NotifyDesigner(Self, Item, opRemove);
Changed; // here
end;</pre></li>
<li>Removing item from the collection will run notifiers - including <code>TContosoEditorPropertiesCollection.Update</code>:<br />
<pre class="brush:delphi">procedure TCollection.Changed;
begin
if FUpdateCount = 0 then Update(nil); // here
end;
procedure TContosoEditorPropertiesCollection.Update(Item: TCollectionItem);
var
I: Integer;
begin
for I := 0 to Count - 1 do
GetItem(I).EditContainer.FCellIndex := I;
Row.EditorsChanged; // here
end;</pre></li>
<li>The Contoso library is trying to update editors:<br />
<pre class="brush:delphi">procedure TContosoCustomMultiEditorRow.EditorsChanged;
begin
if Properties.Locked or VerticalGrid.IsLoading then Exit;
ViewInfo.ClearValuesInfo; // here
Changed;
end;
procedure TContosoCustomRowViewInfo.ClearValuesInfo;
begin
FIsRightToLeftConverted := False;
FInitialized := False;
ValuesInfo.Clear; // here
ValuesLinesInfo.Clear;
end;
procedure TContosoObjectList.Clear;
var
I: Integer;
begin
if OwnObjects then
begin
for I := 0 to Count - 1 do // = 2
FreeItem(I); // here
end;
inherited Clear;
end;</pre></li>
<li>There are 2 items in the <code>ValuesInfo</code> list. First one (index 0) is OK, second one (index 1) is what causing the issue:<br />
<pre class="brush:delphi">destructor TContosoEditCellViewInfo.Destroy;
begin
if (EditContainer <> nil) and not EditContainer.IsDestroying then // here
// ...
end;</pre>
Here is the problem: <code>EditContainer</code> is actually the <code>Owner</code>, and it points to already deleted object - the one that was deleted on the step 2 inside the <code>TContosoCustomEditorRowProperties.Destroy</code>.</li>
</ol>
Short recap:
<ol>
<li>Customer's code deletes an item from the collection;</li>
<li>The item deletes its field;</li>
<li>The collection notifies about item's deletion;</li>
<li>The notification callback tries to clear associated information and accesses the deleted field in process.</li>
</ol>
So, here is our bug: accessing an already deleted object.<br />
<br />
Are you asking how the code was working "flawlessly" before adding EurekaLog to the project? Well, simple: the deleted object remained unchanged in the memory, so the <code>TContosoEditCellViewInfo.Destroy</code> could successfully access the already deleted object and read unchanged data from it. Adding EurekaLog to the project with default settings (enabled memory checks) changes this behaviour by actually erasing deleted object/memory.<br />
<br />
So, is it a bug in the Contoso library? While it certainly does look that way, we can not be certain for sure, as we are not experts with that library. So it is also entirely possible that it is a customer's error.<br />
<br />
Customer reported that this issue will be taken to the Contoso library support.<br />
<br />
P.S. <a href="https://blog.eurekalog.com/search/label/Stories" title="EurekaLog Blog: Stories">Read more stories like this one</a> or <a href="https://www.eurekalog.com/casestudies.php" title="Case Studies and User Reviews">read feedback from our customers</a>.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-68634428445281874462022-07-20T14:15:00.008+01:002023-05-30T08:59:06.745+01:00EurekaLog erases my bitmap? (even VCL has bugs)We were contacted by a client who claimed that EurekaLog was erasing his image in a program. In particular, it was claimed that the client's code "worked great" until EurekaLog was added to the application. After adding EurekaLog, the previously "working" code stopped working, clearing the image instead (e.g. the code's result was blank instead of expected image). The client also claimed that if the "Extended memory manager" option in EurekaLog is disabled - everything works fine again. But if this option is enabled - nothing works, even if all other memory debugging options are disabled.<br />
<br />
Simplified code looks really simple:<br />
<pre class="brush:delphi">Buffer := TBitmap.Create;
try
Buffer.SetSize(64, 64);
Buffer.Canvas.StretchDraw(Rect(0, 0, 64, 64), Bitmap);
Bitmap.SetSize(64, 64);
Bitmap.Canvas.Draw(0, 0, Buffer);
finally
Buffer.Free;
end;</pre>
Do you see a problem in that code?<br />
<a name='more'></a><br />
The client's statement about the "Extended memory manager" option turned out to be not entirely true: the client did not disable the "When memory is released" option - it was set to the "Fill memory with non-zeros" value, although the "disabled" value is the "Do nothing" value. Because of this, I went down the wrong path for quite some time, which I will not describe here.<br />
<br />
So, the correct input data looks like this: the "Extended memory manager" option is enabled; if the "When memory is released" option is set to the "Do nothing" value, then everything works as it should; and if it is set to the "Fill memory with non-zeros" value, then the copied bitmap is erased.<br />
<br />
What does this tell us? Of course, it just screams about an error in accessing data after deleting it (use after free bug)!<br />
<br />
But how do we find it?<br />
<br />
It won't be easy in this case.<br />
<br />
You see, the problem is that some code <b>reads</b> data from an already deleted object (or memory area). But there are no breakpoints for <i>reading</i> data in Delphi. Yes, there is data breakpoints, but those works only for memory writes, not reads. E.g. if some rogue code was writing to deleted object/memory, then we could set a data breakpoint on the memory block and just run the program, waiting for the debugger to stop at the breakpoint and poke our nose into the "bad" code. But we cannot do the same for the memory read operation.<br />
<br />
So what should we do?<br />
<br />
First of all, it would be good to identify the object/memory block that is being accessed after deletion. You can do this with selective wipe.<br />
<br />
Luckily, the code above doesn't delete too many memory blocks, so this can be done in a reasonable amount of time. Run the program, set a breakpoint on the <code>EurekaFreeMem</code> function (yes, you will need the EurekaLog Enterprise edition - which comes with source code, Professional edition does not have source code). Run the code and mark all the places where <code>EurekaFreeMem</code> fires. Sometimes you will also need to set a breakpoint on realloc, but a breakpoint on FreeMem only is sufficient in this case.<br />
<br />
Now run the program again, stop at the first breakpoint. Of course, the "first one" is after your code starts executing, not the first one after the program starts. To do this: the breakpoint must be disabled at program startup and enabled immediately before the problematic code starts executing.<br />
<br />
Next, you need to step through the EurekaLog's code until you see the memory cleanup code. In particular, it is called like this:
<pre class="brush:delphi">function EurekaFreeMem(P: Pointer): Integer;
// ...
begin
// ...
if CheckOurPointer(P, True, True, Error, IsDummy, SharedBlock, Caller) then
// ...
end;
function CheckOurPointer(var P: Pointer; const CheckForFree,
ClearData: Boolean; out Error: TMemoryError; out Dummy: Boolean;
out SharedBlock: Boolean; const AFirstAddr: Pointer = nil): Boolean;
// ...
begin
// ...
if ClearData then
begin
if (loCatchLeaksExceptions in LeaksOptions) and
(PtrDummy^.BlockType = btObject) then
PtrDummy^.Tag := Pointer(P^);
FillReleasedMem(P, PtrDummy^.SingleSize, { ... });
end;
// ...
end;</pre>
E.g. you can set a breakpoint on <code>ClearData</code> or <code>FillReleasedMem</code> instead of <code>EurekaFreeMem</code> (the <code>CheckOurPointer</code> function has two places where <code>FillReleasedMem</code> is called).<br />
<br />
So, now we are at the memory cleanup in the first deleted memory block within our problematic code.<br />
<br />
Open the CPU debugger (View / Debug Windows / CPU Windows / Entire CPU):<br />
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzCIXkLXsGmxn_yqyYyaVWO2g2KZEmcq5Cgcq5rZmlZjBbQSph61wzwxfeM-iA9JOc4f4jF17CdYQrne8EvuNNhiHv2REbVoP6Hjw2N_SxT0LNyTeAvxDq0nBUqfZibDebKR_sHyUs5QjF6VHoL-5kCm5CQDYv5x9M2RXEnLBwXzXR5YfbK8Ks9lzjcg/s1441/%D0%91%D0%B5%D0%B7%D1%8B%D0%BC%D1%8F%D0%BD%D0%BD%D1%8B%D0%B9.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="400" data-original-height="750" data-original-width="1441" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzCIXkLXsGmxn_yqyYyaVWO2g2KZEmcq5Cgcq5rZmlZjBbQSph61wzwxfeM-iA9JOc4f4jF17CdYQrne8EvuNNhiHv2REbVoP6Hjw2N_SxT0LNyTeAvxDq0nBUqfZibDebKR_sHyUs5QjF6VHoL-5kCm5CQDYv5x9M2RXEnLBwXzXR5YfbK8Ks9lzjcg/s400/%D0%91%D0%B5%D0%B7%D1%8B%D0%BC%D1%8F%D0%BD%D0%BD%D1%8B%D0%B9.png"/></a></div>
You can't change the <code>ClearData</code>'s value via the Evaluate / Modify command, because it is a const argument. But you can change its value on the CPU stack - if you know how to do it. Or you can simply execute the compare CPU instruction and then change CPU flags.<br />
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjl017JNIIlyub3aukWUVE4nmzu4USYSLFvJ090JFnd4UJfjwnSe72KBGEThKlAAevMyv0HcnwGHOFp1HwIQ9l93EOo6rQHN5thz2jselBHxQ1xPEjqH-4vU3ZbGcYGsFJAcM0eTLsYnSlzPNUFGwV9xAacW73CImumFJ5OC3wg5ID1duUPxaoApFBkpA/s1255/%D0%91%D0%B5%D0%B7%D1%8B%D0%BC%D1%8F%D0%BD%D0%BD%D1%8B%D0%B9.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="400" data-original-height="359" data-original-width="1255" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjl017JNIIlyub3aukWUVE4nmzu4USYSLFvJ090JFnd4UJfjwnSe72KBGEThKlAAevMyv0HcnwGHOFp1HwIQ9l93EOo6rQHN5thz2jselBHxQ1xPEjqH-4vU3ZbGcYGsFJAcM0eTLsYnSlzPNUFGwV9xAacW73CImumFJ5OC3wg5ID1duUPxaoApFBkpA/s400/%D0%91%D0%B5%D0%B7%D1%8B%D0%BC%D1%8F%D0%BD%D0%BD%D1%8B%D0%B9.png"/></a></div>
In any case - the debugger should indicate that compare has failed and the code below will be skipped:<br />
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6m2Ae7C2DglNAI4L5hqxxKfScj1vZSLrdQ0OoSOScxiVX-3Kwu2UbcN90ONbITYKBmXPAFA7cuddfgw6asI9a4jOt1A0wxRbL8WxrR5SeAT8tVq7oDTmuWpZ4Qz9dsbY383TEdOfJeWd93tOKQpuABgksNG3P9P3TOmZwRjZVsUBCNL3D9RRx3gd0Ow/s1062/%D0%91%D0%B5%D0%B7%D1%8B%D0%BC%D1%8F%D0%BD%D0%BD%D1%8B%D0%B9.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="400" data-original-height="218" data-original-width="1062" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6m2Ae7C2DglNAI4L5hqxxKfScj1vZSLrdQ0OoSOScxiVX-3Kwu2UbcN90ONbITYKBmXPAFA7cuddfgw6asI9a4jOt1A0wxRbL8WxrR5SeAT8tVq7oDTmuWpZ4Qz9dsbY383TEdOfJeWd93tOKQpuABgksNG3P9P3TOmZwRjZVsUBCNL3D9RRx3gd0Ow/s400/%D0%91%D0%B5%D0%B7%D1%8B%D0%BC%D1%8F%D0%BD%D0%BD%D1%8B%D0%B9.png"/></a></div>
Now, continue execution (and disable all breakpoints). We just skipped over wiping the first released memory block. Did it help?<br />
<br />
If not (and in our case - it will not work), then restart the app, repeat everything, but this time skip <b>one</b> <code>EurekaFreeMem</code> call and stop on the second call. Change check's results again to skip the wipe. Does it work now? If not - then restart again, skip <b>two</b> <code>EurekaFreeMem</code> calls and disable wipe on the third call. Repeat this until app starts working. <br />
<br />
Sooner or later, you will find that memory block, clearing which leads to erroneous behavior of the program.<br />
<br />
And in this case, it turned out to be a <code>TBitmapImage</code> with a stack like this:<br />
<br />
<code>Vcl.Graphics.TBitmapImage.Destroy<br />
System.TObject.Free<br />
Vcl.Graphics.TSharedImage.Release<br />
Vcl.Graphics.TBitmap.NewImage<br />
Vcl.Graphics.TBitmap.ReadDIB<br />
Vcl.Graphics.TBitmap.ReadStream<br />
Vcl.Graphics.TBitmap.LoadFromStream<br />
Vcl.Graphics.TBitmap.HandleNeeded<br />
Vcl.Graphics.TBitmap.GetCanvas<br />
Vcl.Graphics.TBitmap.Draw<br />
Vcl.Graphics.TCanvas.StretchDraw</code>
<br />
<br />
The deletion code itself looks like this:<br />
<pre class="brush:delphi">procedure TBitmap.NewImage(NewHandle: HBITMAP; NewPalette: HPALETTE;
const NewDIB: TDIBSection; OS2Format: Boolean; RLEStream: TStream = nil);
// ...
begin
// ...
EnterCriticalSection(BitmapImageLock);
try
FImage.Release; // - it will be deleted here
FImage := Image;
FImage.Reference;
finally
LeaveCriticalSection(BitmapImageLock);
end;
// ...
end;</pre>
Nothing wrong at first glance: the old image is deleted, a new one is inserted instead. But our test data shows that <i>somewhere</i> there is still a reference to the deleted <code>TBitmapImage</code>.<br />
<br />
How do we find it?<br />
<br />
I've been scratching my head, but I couldn't think of anything better than remembering the value of the object's pointer at the moment it was deleted, and then go through the code further, watching where this value will be used.<br />
<br />
To do this, I stopped at the moment of deleting the object and calculated the value of <code>Pointer(Self)</code> via Evaluate / Modify (if you are inside <code>TBitmapImage.Destroy</code>), or <code>Pointer(FImage)</code> (if you are inside <code>TBitmap.NewImage</code>). Let's assume this value is $2A411E0. For convenience, we write it in the full form: $02A411E0.<br />
<br />
Now you need to open the CPU debugger and continue executing the program. You need to watch if this value (or a value close to it, but slightly bigger) appears anywhere in CPU registers.<br />
<br />
This task is compicated by the fact that the code has just created a new <code>TBitmapImage</code> whose value will be close to ours. In our case it will be $02A41000. And it is important for us not to confuse these two values.<br />
<br />
Fortunately, when you do F7/F8 through code in the CPU debugger, it shows you (by highlighting) which registers have been changed by the command you just executed. This makes it a little bit easier to keep track of addresses.<br />
<br />
You will have to execute some code when you first notice our value:<br />
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoM9GzRt5279t3RJfxlyycRdzhrGzhBHmDAUCfvH1Ur9_5-bmCAR8zIaZQhYNIUNyAr3dgYAKM1_CfWa5uiB-w6a4oN_E6wc3xgJH3zcfrqqZVL-jET8zFUOUBSxV3tvx_9HiMrnNwhESSYND8dF4bn-TSzl8B1ULZj27y8OaplxNi_1WOk1ApNFUIRQ/s1129/%D0%91%D0%B5%D0%B7%D1%8B%D0%BC%D1%8F%D0%BD%D0%BD%D1%8B%D0%B9.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="400" data-original-height="169" data-original-width="1129" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoM9GzRt5279t3RJfxlyycRdzhrGzhBHmDAUCfvH1Ur9_5-bmCAR8zIaZQhYNIUNyAr3dgYAKM1_CfWa5uiB-w6a4oN_E6wc3xgJH3zcfrqqZVL-jET8zFUOUBSxV3tvx_9HiMrnNwhESSYND8dF4bn-TSzl8B1ULZj27y8OaplxNi_1WOk1ApNFUIRQ/s400/%D0%91%D0%B5%D0%B7%D1%8B%D0%BC%D1%8F%D0%BD%D0%BD%D1%8B%D0%B9.png"/></a></div>
In this case, the saved CPU registers are restored when exiting the <code>TBitmap.LoadFromStream</code> method.<br />
<br />
Of course, this does not mean that the bug in the code is in the <code>TBitmap.LoadFromStream</code> method. It means that the CPU register still holds a reference to the already-deleted <code>TBitmapImage</code> object, and it is likely that someone higher up in the code is using that register to access (already-deleted) data.<br />
<br />
Let's continue.<br />
<br />
In general, there will be a lot more code. At some point, our value will be overwritten, but then restored again from the one saved on the stack. At the same time, it will always be stored in the <code>ESI</code> register - which, of course, makes it very easy for us to find an error in the code, since we just need to look at machine opcodes to see if any of them accesses the <code>ESI</code> register. And sooner or later we will find such code:<br />
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0Gun6AWREdDVo_zodxb1GgpQ2BLsCQVRvWPoBzaP6OcIfHPaDuocosE50Ab-3j5JAp3NdcX9SGY8rWDxr7q_wgvgkenXgjWm3GYOkndd2M_D361EmoOTKM6QaUyWA1a2BG98Pt52zRUJBfeuEGEFzKq-ln9A7KE9wIKFx3S19T-6bRTKW0_Kr6G890Q/s1131/%D0%91%D0%B5%D0%B7%D1%8B%D0%BC%D1%8F%D0%BD%D0%BD%D1%8B%D0%B9.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="400" data-original-height="252" data-original-width="1131" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0Gun6AWREdDVo_zodxb1GgpQ2BLsCQVRvWPoBzaP6OcIfHPaDuocosE50Ab-3j5JAp3NdcX9SGY8rWDxr7q_wgvgkenXgjWm3GYOkndd2M_D361EmoOTKM6QaUyWA1a2BG98Pt52zRUJBfeuEGEFzKq-ln9A7KE9wIKFx3S19T-6bRTKW0_Kr6G890Q/s400/%D0%91%D0%B5%D0%B7%D1%8B%D0%BC%D1%8F%D0%BD%D0%BD%D1%8B%D0%B9.png"/></a></div>
The code reads a field from an object stored in the <code>ESI</code> register and loads that value into the <code>EAX</code> register. Note that the loaded value is <code>DEADBEEF</code> - this is the debug marker that EurekaLog fills in the memory being deleted. This once again confirms that it is this code that refers to an already deleted object.<br />
<br />
In addition to <code>DEADBEEF</code>, you can also encounter other values: $CCCCCCCC, $FEEEFEEE, $FDFDFDFD, $FADEDEAD, $00009999. EurekaLog attempts to reserve at least one of them for debugging purposes and will use the first one for which the reservation succeeds. There may also be values close to them (i.e. debug marker + offset, e.g. <code>DEADBE8F</code>).<br />
<br />
Besides, it can also be a pointer to the code, following which you will get to the beginning of the <code>DeadObjectVirtualMethodCall</code> or <code>DeadInterfaceMethodCall</code> procedures - it is EurekaLog's service code for catching virtual methods calls to already deleted objects and interfaces. Unfortunately, pointers to <code>DeadObjectVirtualMethodCall</code> or <code>DeadInterfaceMethodCall</code> are not fixed and will be different in each program. Perhaps you should write them down on a piece of paper in advance when you start the program, so that later you can see if they appear in the CPU registers.<br />
<br />
Anyway, we have found the code that accessed the already deleted <code>TBitmapImage</code>:<br />
<pre class="brush:delphi">procedure TBitmap.Draw(ACanvas: TCanvas; const Rect: TRect);
// ...
begin
with Rect, FImage do
begin
// ...
StretchBlt(ACanvas.FHandle, Left, Top, Right - Left, Bottom - Top,
Canvas.FHandle, 0, 0, FDIB.dsbm.bmWidth,
FDIB.dsbm.bmHeight, ACanvas.CopyMode);
// ...
end;
end;</pre>
Do you see a bug in the code now?<br />
<br />
The bug is using the <code>with</code> operator.<br />
<br />
Indeed, using the <code>with</code> operator causes the Delphi optimizer to store a reference to the evaluated expression in <code>with</code> somewhere (in this case - in the <code>ESI</code> register). And since we are talking about an object (which are stored by reference, and not by value), it is not the object's data that is stored, but only a reference to the data. Therefore, any data change (including their deletion/clearing) from the sub-procedures called in the code will go unnoticed. In particular:<br />
<pre class="brush:delphi"> // FImage is being cached by with
with Rect, FImage do
// ...
// FImage was deleted (recreated), but reference to the deleted object is being stored by with
Canvas.RequiredState(csAllValid);
// ...
// FImage.FDIB references the deleted object
StretchBlt(ACanvas.FHandle, Left, Top, Right - Left, Bottom - Top,
Canvas.FHandle, 0, 0, FDIB.dsbm.bmWidth,
FDIB.dsbm.bmHeight, ACanvas.CopyMode);
// ...</pre>
And the worst thing about this <code>with</code> operator is that the debugger is not able to interpret it correctly. For example, the calculation of expressions by mouse hovering does not work. And if we try to calculate the values of the specified expressions (for example, <code>FImage.FDIB.dsbm.bmWidth</code>), then the debugger will, of course, return the correct value from the <i>new</i> <code>FImage</code> object, and not from the removed old one, which is preserved by the <code>with</code> statement.<br />
<br />
Values of the <code>StretchBlt</code>'s function argument were one of the things that I have checked first! But due to this "feature" of the debugger, I could not see the problem, although I had it right before my eyes. I had to go the long way.<br />
<br />
Conclusion: customer has found a bug in VCL. Congratulations!<br />
<br />
But what the customer should do? After all, fixing an error in the VCL can take a long time, and the program should work like right now.<br />
<br />
Luckily for him, a workaround can be suggested in this case. Basically, we need to move the <code>FImage</code> re-creation outside of the <code>with</code> block. For example:<br />
<pre class="brush:delphi">Buffer := TBitmap.Create;
try
Bitmap.Canvas; // - added
Buffer.SetSize(64, 64);
Buffer.Canvas.StretchDraw(Rect(0, 0, 64, 64), Bitmap);
Bitmap.SetSize(64, 64);
Bitmap.Canvas.Draw(0, 0, Buffer);
finally
Buffer.Free;
end;</pre>
If you can't offer a workaround, then what is left is disabling memory checks in EurekaLog.<br />
<br />
Moral of the story:<br />
<ul>
<li>Do not use the <code>with</code> operator;</li>
<li>If you have a memory error somewhere in the code that uses <code>with</code> - refactor the code so it will not use <code>with</code>;</li>
<li>VCL can have bugs too;</li>
<li>If there are errors in your program after adding EurekaLog to it - try disabling memory checks in EurekaLog. If the problem goes away after that - 99% that your program has a memory bug somewhere.</li>
</ul>
<br />
P.S. You can vote for fixing this issue here: <a href="https://quality.embarcadero.com/browse/RSP-38694" title="[RSP-38694] Using with in VCL causes use after free bugs">[RSP-38694] Using "with" in VCL causes "use after free" bugs</a>.<br />
P.P.S. It is possible that VCL has <a href="https://blog.eurekalog.com/2021/02/EurekaLog-causes-EOutOfResources.html" title="EurekaLog causes EOutOfResources (Out of system resources)?">another similar bug</a>.<br />
<br />
P.P.S. <a href="https://blog.eurekalog.com/search/label/Stories" title="EurekaLog Blog: Stories">Read more stories like this one</a> or <a href="https://www.eurekalog.com/casestudies.php" title="Case Studies and User Reviews">read feedback from our customers</a>.
GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-11736249535126861192022-05-16T09:29:00.002+01:002022-07-18T18:45:59.504+01:00Using EurekaLog with GMail (Google Mail) after May 30, 2022Google has announced that it’s disabling the <a href="https://support.google.com/accounts/answer/6010255" title="Less secure apps & your Google Account">Less Secure Apps feature</a> on some Google accounts from May 30th, 2022.
<blockquote>To help keep your account secure, starting May 30, 2022, Google will no longer support the use of third-party apps or devices which ask you to sign in to your Google Account using only your username and password.</blockquote>
<a name='more'></a>
Google Workspace and Google Cloud Identity users won’t be affected right away, but support for Less Secure Apps will still be phased out for those users at a later date.<br />
<ul>
<li>This change will affect you if you are using EurekaLog to send bug reports via GMail while <b>using your GMail password</b>. This method is considered unsecure and <b><font color="red">will no longer work on most accounts starting May 30, 2022</font></b>.</li>
<li>This change will <b>not</b> affect you if you are using EurekaLog to send bug reports via GMail while using application-specific passwords.</li>
</ul>
While we’ve <a href="https://www.eurekalog.com/help/eurekalog/tracker_security.php" title="Security Considerations">recommended using application-specific passwords instead of your GMail passwords</a> for quite some time, but some customers have chosen to use the "Less Secure Apps" feature since it’s a little more straightforward.<br />
<br />
In other words:
<ol>
<li>If you don't already have 2-Step Verification (also known as two-factor authentication) enabled for your account - you must <a href="https://support.google.com/accounts/answer/185839" title="Turn on 2-Step Verification">turn it on</a>. (Application-specific passwords can only be used with accounts that have 2-Step Verification turned on.) Once enabled - you will have to confirm each sign in using your GMail password via a promt on already signed device, a physical security key, a code from a verification app, a phone call/SMS, or a backup code.</li>
<li>(Optional) Turn off/disable the "Less secure app access" feature.</li>
<li>You should <a href="https://support.google.com/accounts/answer/185833" title="Sign in with App Passwords">create an application-specific password</a> for your application with EurekaLog. An App Password is a 16-digit (random) passcode that gives your app permission to access your Google Account.
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgR0BZ93nzwceklhaR3xEMwze8QFv0SufWd2c-fglzcflR59HHc_sR86RJFigz5TPWgN4k34eu9Md0XzSvVrKbOGhltZps_xwQaIQQVpdBu44zrJT_B2Q1ZREuHgeXYU077BF5TTfZ_vRmUfrmoLssNQjVgYvUUS9WBOQJhxSTV31b-AI7yWbLip38s/s1116/generated-app-password.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="400" data-original-height="995" data-original-width="1116" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgR0BZ93nzwceklhaR3xEMwze8QFv0SufWd2c-fglzcflR59HHc_sR86RJFigz5TPWgN4k34eu9Md0XzSvVrKbOGhltZps_xwQaIQQVpdBu44zrJT_B2Q1ZREuHgeXYU077BF5TTfZ_vRmUfrmoLssNQjVgYvUUS9WBOQJhxSTV31b-AI7yWbLip38s/s400/generated-app-password.png"/></a></div>
Basically, you can use App Password instead of your (main/primary) GMail password. Most of the time, you’ll only have to enter an App Password once (per application), so don’t worry about memorizing it. You can always create a new/additional one.</li>
</ol>
<br />
Please note that we also recommend to use web-based bug tracker instead of e-mails for many reasons:<br />
<ul>
<li><a href="https://www.eurekalog.com/help/eurekalog/reporting.php" title="Reporting">Using the large number's law and Exception Driven Development - EDD</a>.</li>
<li>Using HTTP(S) typically avoids all pitfalls with firewalls and security.</li>
<li>Finer control over access rights.</li>
<li>Better UI and report management.</li>
<li>Etc...</li>
</ul>
<br />
P.S. There are <a href="https://support.google.com/a/answer/176600" title="Send email from a printer, scanner, or app">two other alternative methods</a>:
<ol>
<li>SMTP relay service</li>
<li>Restricted Gmail SMTP server</li>
</ol>GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-27306113138692477632022-05-11T12:30:00.002+01:002022-07-18T18:45:43.976+01:00Why it is recommended to restart the application after a crashWe were contacted by a client complaining that EurekaLog was hiding his application.<br />
<a name='more'></a><br />
In particular, the client was trying to make the error dialog to appear only for certain exceptions. He tried to do this with the following code:<br />
<pre class="brush:delphi">if AShowDlg then
EI.Options.ExceptionDialogType := edtEurekaLog
else
EI.Options.ExceptionDialogType := edtNone;</pre>
and was testing it like this:<br />
<pre class="brush:delphi">procedure TForm1.Button1Click(Sender: TObject);
var
List: TStringList;
begin
try
List.Add('Test');
except
on E: Exception do
HandleError(E);
end;
end;</pre>
where <code>HandleError</code> is the client's code for handling exceptions.<br />
<br />
At the same time, the client claimed that "everything works correctly without EurekaLog", and "with EurekaLog: after processing the test exception, (another) Access Violation is raised, and the application disappears from the monitor, while still showing in the task manager".<br />
<br />
Do you see a problem in the above code?<br />
<br />
The problem is that the test case uses an uninitialized <code>List</code> variable. This means that it can be <b>any</b> value (which will be an arbitrary garbage from the stack left after execution of the previous code).<br />
<br />
It just so happened (read: "lucky") that in an application without EurekaLog, this garbage pointed to an invalid memory area, so an attempt to call the <code>List.Add</code> threw an Access Violation exception immediately when the code tried to call the method. The message looked like this: 'Access violation at address 00000001 in module 'Project1.exe'. Read of address 00000001'. Note that both addresses match and are invalid.<br />
<br />
However, when EurekaLog was added to the application, the situation changed, and a different garbage value appeared on the stack. It just so happened (read: "unlucky") that this garbage pointed to a memory area in which the address of another method was written. We don't know what this value was on the client's machine, but when we checked, the value in memory pointed to the <code>TCustomForm.Create</code>. Therefore, the <code>List.Add</code> call actually called the <code>TCustomForm.Create</code> - with a junk <code>Self</code> (pointing to the main form) and junk arguments. Of course, this cannot work correctly. The VCL code started executing, used garbage from the arguments, and eventually crashed when trying to register the created form with a non-existent Owner. The message looked like this: 'Access violation at address 004092F9 in module 'Project1.exe'. Read of address E8C78BD6'. Note that the addresses are different; and the first address is correct, but the second is not.<br />
<br />
EurekaLog has catched this exception (which the client mistook for an exception when trying to call the <code>List.Add</code>) and handled it correctly. However, the VCL was already in a corrupted state. Namely, when the EurekaLog dialog was shown (or after it was closed), the message loop accessed the saved garbage data, which led to the second ("incorrect") Access Violation exception (which EurekaLog also caught).<br />
<br />
The application "disappeared" for the reason that the main form was "overwritten" when it was passed as <code>Self</code> in <code>TCustomForm.Create</code>.<br />
<br />
That is why, if you want to test throwing exceptions, then you should do it like this:<br />
<pre class="brush:delphi">List := nil; // - added
List.Add('Test');</pre>
<br />
This example also perfectly illustrates why you should restart or close your application immediately after handling an exception: your application may be in a invalid state. Indeed, if it were in the correct state, then the unexpected exception would not have occurred. And once something unexpected happened, the application is no longer in the expected state. This means that its further behavior is undefined. An attempt to continue execution can lead to data corruption, the throwing of other (extremely difficult to diagnose) exceptions, and so on.<br />
<br />
Of course, it is extremely important to separate really "unexpected" exceptions from "expected" ones. Unfortunately, this is a vast topic that is beyond the scope of this post.<br />
<br />
<a href="https://blog.eurekalog.com/search/label/Stories" title="EurekaLog Blog: Stories">Read more stories like this one</a> or <a href="https://www.eurekalog.com/casestudies.php" title="Case Studies and User Reviews">read feedback from our customers</a>.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-7256969321046700402022-01-24T20:00:00.000+00:002022-01-24T20:00:10.220+00:00EurekaLog 7.10 is outWe are pleased to announce the availability of the new 7.10 version.<br />
<a name='more'></a><br />
<div style="text-align: center;">
<a href="https://www.eurekalog.com/" target="_blank" title="EurekaLog Logo"><img alt="EurekaLog Logo" src="https://www.eurekalog.com/images/logo.png" height="68" width="295" /></a></div>
<blockquote>
EurekaLog is a tool for Delphi and C++Builder that gives your application the power to catch every exception and memory/resource leak, generating a detailed log of the call stack (with unit, procedure and line number), optionally sending you a copy of each log entry via email or the Web via the most used Web Bug-Tracking tools (as Mantis, BugZilla, FogBugz, JIRA, YouTrack, Redmine, Exceptionless, GitLab, and GitHub).</blockquote>
EurekaLog represents the most advanced exception and memory leaks logger technology available for Delphi/C++Builder developers.<br />
<br />
To learn more about EurekaLog, please visit our website at:<br />
<a href="https://www.eurekalog.com/" target="_blank" title="https://www.eurekalog.com">https://www.eurekalog.com</a><br />
<br />
To download a demo, please visit:<br />
<a href="https://www.eurekalog.com/downloads.php" target="_blank" title="https://www.eurekalog.com/downloads.php">https://www.eurekalog.com/downloads.php</a><br />
<br />
If you would like to purchase a new license, please visit:<br />
<a href="https://www.eurekalog.com/buy.php" target="_blank" title="https://www.eurekalog.com/buy.php">https://www.eurekalog.com/buy.php</a><br />
<br />
<strong>Changes in 7.10 build:</strong><br />
<ol>
<li>Added: Support for RAD Studio 11</li>
<li>Added: Windows 11 detection</li>
<li>Added: GitLab support</li>
<li>Added: GitHub support</li>
<li>Added: New tool: Crypto Helper</li>
<li>Added: "Use inner exception" option to exception filters</li>
<li>Added: Support for buffered logging. Default is OFF. Can be enabled via --el_debug_buffering command line switch or when using the new ABufferSize argument in the ELogOpen function. Use the new ELogFlush function to flush buffer to disk before exiting process/thread</li>
<li>Added: uROCOMInit to PreInitUnits</li>
<li>Added: Additional protection for passwords inside executables and options</li>
<li>Added: EurekaLog's configurations now have desriptions. You can enter descriptions for custom configs when saving/exporting options</li>
<li>Added: UI option to disable EurekaLog on RAD Studio 2007 and earlier (as base config only, will not work for profiles)</li>
<li>Added: Ability (option) to partially match exception message in exception filters</li>
<li>Added: More public crypto functions</li>
<li>Added: If bug tracker sender will fail to upload the file - the comment with error message will be added</li>
<li>Added: EBase.SetExceptionMessage helper routine to change exception message in exception object as well as in EurekaLog</li>
<li>Added: atEurekaLogInitializing / atEurekaLogInitialized action types for OnExceptionActionEvent</li>
<li>Added: Support for CC and BCC special headers to SMTP Client</li>
<li>Added: More info is extracted for MS C++ exceptions</li>
<li>Added: More info for stack overflow exceptions</li>
<li>Added: CanCallMemWipeStack function, now MemWipeStack will not be called when it is not safe (this also fixes rare crashes in unconventional environments)</li>
<li>Added: atExceptionHandling / atExceptionHandled action types</li>
<li>Added: ThreadStackGuarantee option, EnsureThreadStack function, _resetstkoflw function</li>
<li>Added: New options for HTTP upload send method</li>
<li>Added: "Delete bug report file at startup" option</li>
<li>Added: Merging includes/generics - as workaround for excessive info in latest IDEsv
<li>Added: Removing invalid rogue line entries - as workaround for linker bugs</li>
<li>Added: Convenient wrappers around EurekaLog functions: __MODULE__, __UNIT__, __FILE__, __FUNCTION__, __LINE__ (EDebugInfo); __DATE__, __TIME__ (EModules)</li>
<li>Added: Command-line arguments for ManageProfiles.exe tool</li>
<li>Added: Workaround for RSP-31458</li>
<li>Added: New "GDI objects", "USER objects", "kernel handles" fields in crash reports</li>
<li>Added: Support for SysUtils.ResStringLoad (RAD Studio 10.4)</li>
<li>Added: Support for {$LIBSUFFIX AUTO}</li>
<li>Added: Now EurekaLog would allow you to copy entire bug report if no text is selected in dialogs, and copy selected text only if there is something selected.</li>
<li>Added: [Viewer] Added option for switching between time zones</li>
<li>Added: [Viewer] "View in new instance" option for context menu of bug reports - to open report in new standalone instance of the Viewer</li>
<li>Fixed: More customizations for themes. EConsts now have global vars (look for ECol... and color_...), TBaseDialog have new virtual methods. Now EurekaLog is compatible with VCL Styles Utils library by RRUZ. You can add the following units after ExceptionLog7,{$ENDIF}: Vcl.Styles.Ext, Vcl.Styles.Utils, Vcl.Styles.Fixes, Vcl.Styles.Hooks, Vcl.Styles.ColorTabs, Vcl.Styles.FormStyleHooks, Vcl.Styles.Utils.SysControls, Vcl.Styles.Utils.SysStyleHook, Vcl.Styles.Utils.Menus, Vcl.Styles.WebBrowser, Vcl.Styles.ControlColor, Vcl.Styles.Utils.ComCtrls, Vcl.Styles.Utils.StdCtrls</li>
<li>Fixed: Various fixes for working in low integrity applications</li>
<li>Fixed: Possible memory leak in multi-threading apps in RAD Studio 11 Alexandia</li>
<li>Fixed: Very rare hang on startup (LookupAccountSid may hang for offline/unavailable domains)</li>
<li>Fixed: Compatibility issues in the YouTrack API</li>
<li>Fixed: Exception filter changes exception message in bug report, while it should do this only for dialogs</li>
<li>Fixed: Rare case of two EurekaLog error dialogs (within the same process) fighting over top most window</li>
<li>Fixed: Removed hard-coded colors in dialogs</li>
<li>Fixed: Added workaround for RSP-35509/RSP-36171</li>
<li>Fixed: Various improvements for GetIt installation</li>
<li>Fixed: Various minor UI improvements</li>
<li>Fixed: Issues with exception filters saving</li>
<li>Fixed: Cues in options dialog on ANSI IDEs</li>
<li>Fixed: Rare wrong error message for failed dialog</li>
<li>Fixed: Very rare range check error</li>
<li>Fixed: Access violation in ecc32 when injection of debug info is disabled</li>
<li>Fixed: Possible leak when processing non-Delphi exceptions</li>
<li>Fixed: Global exception counter was off by 1</li>
<li>Fixed: Better User-Agent HTTP header (also added virtual UserAgent function for user overloads)</li>
<li>Fixed: Added few workarounds for anti-virus locking files ("Cannot create file")</li>
<li>Fixed: Improved behaviour for automatic restarts</li>
<li>Fixed: User e-mail will no longer be copied from license e-mail for Company and Corporate licenses</li>
<li>Fixed: Rare crash when retrieving Windows user's email</li>
<li>Fixed: Expected exceptions will no longer register towards global exception counter (also disables auto-crash logic for expected exceptions)</li>
<li>Fixed: Minor UI improvements in project options dialog</li>
<li>Fixed: Significant post-processing performance boost for huge projects with large amount of generics</li>
<li>Fixed: "Restart/terminate after N errors" option now will be reflected by dialog</li>
<li>Fixed: E-mail and reproduce controls were hidden if send consent was disabled</li>
<li>Fixed: DumpAllocationsToFile may sometimes fail</li>
<li>Fixed: Rare crash when using RAW memory stack tracing</li>
<li>Fixed: Possible IDE crash when you manually unload EurekaLog's IDE expert</li>
<li>Fixed: Possible variant conversion error when opening project options (old IDEs)</li>
<li>Fixed: EurekaLog project options dialog now will shrink to fit on small desktops</li>
<li>Fixed: Updating installer to be DPI awared</li>
<li>Fixed: Minor improvements for internal crash reports</li>
<li>Fixed: More precise address for "raise at" constructs</li>
<li>Fixed: Minor improvements and polishing for run-time errors / Invalid Pointer</li>
<li>Fixed: Behaviour for memory checks now respects use safe mode option</li>
<li>Fixed: .map parsing for modified or 3rd party .map files</li>
<li>Fixed: Possible crash in RAW tracing for memory leaks</li>
<li>Fixed: Bug in offsets calculation when setting low-level hooks (rare crash)</li>
<li>Fixed: EurekaLog may fail to create a temp file when you are switching user in a thread (impersonation)</li>
<li>Fixed: [Win64] Building call stacks starting with invalid location</li>
<li>Fixed: [Win64] Possible conflict between two instances of EurekaLog in two different modules</li>
<li>Fixed: [C++ Builder] Various issues with .map parsing (mostly for 64-bit/LLVM)</li>
<li>Fixed: [C++ Builder] Possible crash when using EMemLeaksBCB.cpp in 64-bit</li>
<li>Fixed: [C++ Builder] Adding/removing EurekaLog when working with custom configs from sub-profiles</li>
<li>Fixed: [C++ Builder] Issues with removing EL's obj files from C++ projects when EUREKALOG conditional symbol is not defined; the old behaviour can be enabled via hidden CBuilderSwitchUnits option</li>
<li>Fixed: [C++ Builder] Crash when reporting memory errors after EurekaLog shutdowns</li>
<li>Fixed: [Viewer] Very rare access violation</li>
<li>Fixed: [Viewer] Possible hang on startup if default printer is unavailable network printer</li>
<li>Fixed: [Viewer] Updated auto-downloader to use latest TLS</li>
<li>Fixed: [Viewer] Recompiled with latest DevExpress, fixes various DevExpress issues</li>
<li>Fixed: Various minor improvements</li>
<li>Changed: Injected options are now packed by default even if debug information is not</li>
<li>Changed: Now Viewer will view reports in new window by default. This could be changed back in Viewer's settings</li>
<li>Changed: Minor change in compilation statistics in ecc32 output</li>
<li>Removed: Importing old EurekaLog 6 settings</li>
</ol>GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-17207765151468837982021-11-17T21:00:00.009+00:002021-11-17T21:00:00.188+00:00Black Friday and Cyber Monday SaleWe are pleased to announce Black Friday and Cyber Monday <b>30% discount</b> on any of our EurekaLog products using the coupon code found below.<br />
<br />
The sale starts on Black Friday (November, 26) and ends at the end of Cyber Monday (November, 29).<br />
<br />
Enter this code when paying for the item on our web site:<br />
<br />
<b>BFCM2021</b><br />
<br />
<hr />
<br />
Existing customers with valid or expired licenses can log in and purchase upgrades, new licenses and extensions here:<br />
<br />
<a href="https://www.eurekalog.com/login.php" title="Log in to customer control panel">https://www.eurekalog.com/login.php</a><br />
<br />
Use the login credentials we sent you at purchase time.<br />
<br />
<hr />
<br />
New customers (without existing licenses) can use the discount code here:<br />
<br />
<a href="https://www.eurekalog.com/buy.php" title="Purchase EurekaLog">https://www.eurekalog.com/buy.php</a>GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-90909805240460937922021-09-09T10:06:00.005+01:002021-11-11T14:38:54.357+00:00Support for RAD Studio 11 Alexandria is available in EurekaLogWe are pleased to announce the availability of support for RAD Studio 11 Alexandria in EurekaLog.<br />
<a name='more'></a><br />
<div style="text-align: center;">
<a href="https://www.eurekalog.com/" target="_blank" title="EurekaLog Logo"><img alt="EurekaLog Logo" src="https://www.eurekalog.com/images/logo.png" height="68" width="295" /></a></div>
<blockquote>
EurekaLog is a tool for Delphi and C++Builder that gives your application the power to catch every exception and memory/resource leak, generating a detailed log of the call stack (with unit, procedure and line number), optionally sending you a copy of each log entry via email or the Web via the most used Web Bug-Tracking tools (as Mantis, BugZilla, FogBugz, JIRA, Redmine, and Exceptionless).</blockquote>
EurekaLog represents the most advanced exception and memory leaks logger technology available for Delphi/C++Builder developers.<br />
<br />
Support for RAD Studio 11 Alexandria RTM is available since EurekaLog 7.9 update 5 (7.9.5.0). If you have already downloaded/installed EurekaLog 7.9.5.0 (before 2021-09-04 19:00 UTC) - please, simply download the installer again.<br />
<br />
Existing customers can download new builds from <a href="https://www.eurekalog.com/login.php" target="_blank" title="EurekaLog Client Control Panel">control panel</a>.<br />
New users can <a href="https://www.eurekalog.com/downloads.php" target="_blank" title="EurekaLog Downloads">download trial</a>.<br />
<br />
To learn more about EurekaLog, please visit our website at:<br />
<a href="https://www.eurekalog.com/" target="_blank" title="EurekaLog Web Site">https://www.eurekalog.com</a><br />
<br />
If you would like to purchase a new license, please visit:<br />
<a href="https://www.eurekalog.com/buy.php" target="_blank" title="Purchase EurekaLog License">https://www.eurekalog.com/buy.php</a>GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-56398181656903253682021-07-27T11:32:00.003+01:002022-04-06T14:15:45.085+01:00Working with cryptography in EurekaLog<a href="https://www.eurekalog.com/" title="EurekaLog">EurekaLog</a> is an exception tracer, i.e. a tool that installs hooks and catches exceptions being thrown, allowing you to generate a report on unhandled exceptions. However, it does provide various kinds of additional functionality that you can use in your apps. And one of those features is cryptography functions.<br />
<a name='more'></a><br />
EurekaLog offers 3 units:<br />
<ol>
<li><a href="https://www.eurekalog.com/help/eurekalog/topic_unit_eencoding.php" title="EEncoding Unit"><code>EEncoding</code></a> - contains <a href="#t1">data encoding and transformation functions</a>;</li>
<li><a href="https://www.eurekalog.com/help/eurekalog/topic_unit_ehash.php" title="EHash Unit"><code>EHash</code></a> - contains <a href="#t2">hash functions</a>;</li>
<li><a href="https://www.eurekalog.com/help/eurekalog/topic_unit_eencrypt.php" title="EEncrypt Unit"><code>EEncrypt</code></a> - contains <a href="#t3">symmetric and asymmetric encryption functions</a>.</li>
</ol>
Although the functions from these units will not be able to fully replace a proper cryptographic support library, it may be enough for you in some special cases.<br />
<br />
<font color="red"><b>Important Note:</b></font> please update EurekaLog to the most recent version. Not all features described here are available in previous versions, because some features were published specifically for this article.<br />
<br />
<a name="t1"></a><h1>Encoding</h1>
Before talking about cryptography, you need to take decicion about data representation. For example, suppose you want to get the MD5 hash of the <code>'Привет'</code> string (means "Hello" in Russian, reads as "Privet", stress on the second syllable). What exactly are you gonna feed into hash function? $CF$F0$E8$E2$E5$F2 bytes? (which is <code>'Привет'</code> encoded via ANSI/Windows-1251) Or $1F$04$40$04$38$04$32$04$35$04$42$04 bytes? (<code>'Привет'</code> in Unicode/UTF-16) Or may be $D0$9F$D1$80$D0$B8 bytes (<code>'Привет'</code> in UTF-8)? Depending on how you answer this question, you will get different results. For example, the MD5 hash for the <code>'Привет'</code> in UTF-16 would be 8EFA2364EE560EE1B862ECC8D430C9AD, for <code>'Привет'</code> in ANSI - 43A3F987A7AF93811B7682E43ED0752A, and for <code>'Привет'</code> in UTF-8 - 8A669E9418750C81AB90AE159A8EC410.<br />
<br />
Questions like this probably don't matter if you use cryptography functions exclusively inside your own apps. But as soon as you need to interact with other code - you immediately would have problems with the exact definition of the data.<br />
<br />
Therefore, when you want exact result, you should <b>operate on bytes</b>, not strings. In Delphi, to operate on bytes, you can:<br />
<ul>
<li>Use pointer + size of data: <code>(const ABuffer: Pointer; const ABufferSize: Cardinal)</code>;</li>
<li>Use <a href="http://docwiki.embarcadero.com/Libraries/en/System.SysUtils.TBytes" title="SysUtils.TBytes"><code>TBytes</code></a> (<code>array of Byte</code> - dynamic byte array);</li>
<li>Use <a href="http://docwiki.embarcadero.com/Libraries/en/System.RawByteString" title="System.RawByteString"><code>RawByteString</code></a>;</li>
<li>Use <a href="http://docwiki.embarcadero.com/Libraries/en/System.Classes.TStream" title="Classes.TStream"><code>TStream</code></a> (its sub-classes).</li>
</ul>
<br />
Specifically, EurekaLog functions accepts pointer+size, as well as overloaded option for <code>RawByteString</code>.<br />
<br />
For example, if you try to obtain MD5-hash from "just" string <code>'Привет'</code> in PHP - you would get 8a669e9418750c81ab90ae159a8ec410 - i.e. MD5-hash of UTF-8 encoded <code>'Привет'</code>.<br />
<blockquote>From where you can also conclude that strings in PHP are stored in UTF-8; for comparison: Delphi stores strings as UTF-16 (since Delphi 2009) or ANSI (Delphi 2007 and earlier).</blockquote>
If you want to change the encoding in PHP, you will need to call something like <a href="https://www.php.net/manual/function.mb-convert-encoding.php" title="PHP: mb_convert_encoding"><code>mb_convert_encoding</code></a>. And if you want to change the encoding in Delphi, you need Delphi encoding functions. Specifically, to <a href="http://docwiki.embarcadero.com/RADStudio/en/UTF-8_Conversion_Routines" title="UTF-8 Conversion Routines">convert to/from UTF-8</a>, <a href="http://docwiki.embarcadero.com/Libraries/en/System.SysUtils.TEncoding" title="TEncoding Class"><code>TEncoding</code></a>. In Delphi 2009 and up, you can also just declare the string type of the desired encoding and <a href="http://rvelthuis.de/articles/articles-pchars.html#d2009" title="Строки в Delphi 2009">string data conversion will be done automatically when assigned</a>.<br />
<br />
The same is true in the opposite direction: the result of calling cryptographic functions is a set of bytes (hash, encrypted data, etc.). If you want to display these bytes to a human, you have to convert it to a string. It can be done, again, in different ways. For example, you can use the built-in function <a href="http://docwiki.embarcadero.com/Libraries/en/System.Classes.BinToHex" title="Classes.BinToHex"><code>BinToHex</code></a> or its more convenient equivalents: <code>HexEncodeString</code>/<code>HexEncodeToString</code> from EurekaLog. You can use <code>Base64EncodeString</code>/<code>Base64EncodeToString</code> from EurekaLog. If, suddenly, you need to convert data from/to <code>RawByteString</code>, then EurekaLog has <code>RAWToString</code>/<code>RAWFromString</code> helpers. Also you may want to load/save small data directly to files - there is <code>FileToString</code>/<code>StringToFile</code> for that (from the <code>ECompatibility</code> unit).<br />
<br />
Examples of using the mentioned functions can be found below.<br />
<br />
<br />
<a name="t2"></a><h1>Hashing</h1>
EurekaLog has functions for calculating the following hashes:
<ul>
<li><a href="https://en.wikipedia.org/wiki/Cyclic_redundancy_check" title="Wikipedia: Cyclic redundancy check">CRC16</a></li>
<li><a href="https://en.wikipedia.org/wiki/Cyclic_redundancy_check" title="Wikipedia: Cyclic redundancy check">CRC32</a></li>
<li><a href="https://en.wikipedia.org/wiki/MD5" title="Wikipedia: MD5">MD5</a></li>
<li><a href="https://en.wikipedia.org/wiki/SHA-1" title="Wikipedia: SHA-1">SHA-1</a></li>
<li><a href="https://en.wikipedia.org/wiki/SHA-2" title="Wikipedia: SHA-2">SHA-256</a></li>
<li>SDBM is a good general purpose hash function with uniform distribution, convenient for use as a key/index in a database</li>
</ul>
All hashing functions have name like <i>HashName</i>Hash (for example, <code>MD5Hash()</code>), returns result of <code>T<i>HashName</i>Hash</code> type (for example, <code>TSHA1Hash</code>), and accepts <code>RawByteString</code> on input, as well as pointer+size (overloaded option).<br />
<br />
Additionally, EurekaLog has <a href="https://en.wikipedia.org/wiki/HMAC" title="Wikipedia: Hash-based Message Authentication Code">HMAC</a> implementation for some hashes. One way to use HMAC is to authenticate a user by combining a <a href="https://en.wikipedia.org/wiki/Salt_(cryptography)" title="Wikipedia: Salt (cryptography)">salt</a> and a password to obtain hash via HMAC. HMAC functions have names like <i>HashName</i>HMAC (for example, <code>MD5HMAC()</code>) and accepts password and salt on input.<br />
<br />
Here are some practical examples:<br />
<br />
<b>1. Calculate hash of a string:</b><br />
<pre class="brush:delphi">uses
EEncoding, // for HexEncodeToString
EHash; // for MD5Hash
procedure TForm1.Button1Click(Sender: TObject);
var
S: String; // Source string
UTF8Str: UTF8String; // Byte representation of a string
Hash: TMD5Hash; // Result
begin
// Define source data
S := 'Привет';
// Define exact representation as bytes
// We use UTF-8 in this example
UTF8Str := UTF8Encode(S);
// (you can also just do UTF8Str := S; in Delphi 2009 and up)
// Calculate hash from bytes
Hash := MD5Hash(UTF8Str);
// Show hash to a human
Label1.Caption := HexEncodeToString(@Hash, SizeOf(Hash));
// Should display '8A669E9418750C81AB90AE159A8EC410'
end;</pre>
<br />
<b>2. Calculate hash of a file:</b><br />
<pre class="brush:delphi">uses
EEncoding, // for HexEncodeToString
EHash, // for SHA256Hash
ECompatibility; // for FileToString
procedure TForm1.Button1Click(Sender: TObject);
var
Content: RawByteString; // File's bytes
Hash: TSHA256Hash; // Result
begin
// Loads entire file into memory
Content := FileToString(ParamStr(0));
// Content will be something like 'MZP'#0#2#0#0#0...
// Calculate hash from bytes
Hash := SHA256Hash(Content);
Finalize(Content); // optional
// Show hash to a human
Label1.Caption := HexEncodeToString(@Hash, SizeOf(Hash));
// Should be something like 'FCF52FDC753E3797FE5EE4B5A7680E656D044D6BF7D97C408F0F7874492E43C2'
end;</pre>
<br />
<b>3. Calculate hash of a string in an arbitrary encoding:</b><br />
<pre class="brush:delphi">uses
EEncoding, // for HexEncodeToString (also for TEncoding for older Delphi)
EHash; // for CRC32Hash
procedure TForm21.Button1Click(Sender: TObject);
var
S: String; // Source string
Encoding: TEncoding; // Encoding to encode the string
Content: TBytes; // String's bytes
Hash: TCRC32Hash; // Result
begin
// Define source data
S := 'Привет';
// Define the encoding
Encoding := TEncoding.GetEncoding(866);
// You can also do:
// Encoding := TEncoding.UTF8;
// Encoding := TEncoding.Unicode;
// Encoding := TEncoding.ANSI;
try
// Convert string (characters) to bytes
Content := Encoding.GetBytes(S);
finally
FreeAndNil(Encoding);
end;
// Calculate hash from bytes
Hash := CRC32Hash(Pointer(Content), Length(Content));
Finalize(Content); // optional
// Show hash to a human
Label1.Caption := HexEncodeToString(@Hash, SizeOf(Hash));
// Should be '6DB3A7B9'
// You can also do IntToStr(Hash) - which will be 3114775405
end;</pre>
<br />
<b>4. Check hash in PHP:</b><br />
<pre class="brush:delphi">uses
EEncoding, // for HexEncodeToString
EHash, // for MD5Hash
ECore; // for ShellExec
procedure TForm1.Button1Click(Sender: TObject);
var
S: String; // Source string
UTF8Str: UTF8String; // String's bytes
Hash: TMD5Hash; // Hash (bytes)
HashStr: String; // Hash (text)
begin
// Define source data
S := 'Привет';
// Define exact representation as bytes
// We use UTF-8 in this example
UTF8Str := UTF8Encode(S);
// (you can also just do UTF8Str := S; in Delphi 2009 and up)
// Calculate hash from bytes
Hash := MD5Hash(UTF8Str);
// Convert bytes to text
HashStr := HexEncodeToString(@Hash, SizeOf(Hash));
// Will be '8A669E9418750C81AB90AE159A8EC410'
// Pass hash as text into PHP script
ShellExec(Format('http://localhost/test.php?hash=%s', [HashStr]));
end;</pre>
<pre class="brush:php"><?php
// Source data (stored as UTF-8)
$Source = 'Привет';
// Calculate hash of source (UTF-8 encoded) data
// (the function will return text, not bytes)
$Hash = md5($Source);
// Read script's parameter
$HashArg = $_GET['hash'];
// Ensure both passed and calculated hashes match by comparing string representation
if (strtolower($Hash) == strtolower($HashArg)) {
// or you can do this starting with PHP 5.6:
// if (hash_equals($HashArg, $Hash)) {
echo('OK'); // we should get there,
// e.g. source strings match
// 'Привет' in Delphi = 'Привет' in PHP
} else {
echo('FAIL');
}</pre>
<br />
<b>5. Storing user credentials in a database:</b><br />
<pre class="brush:delphi">uses
EHash, // for SHA256HMAC
EEncrypt, // for InitSalt
EEncoding; // for RAWToString and HexEncodeToString/HexDecodeFromString
procedure TForm1.Button1Click(Sender: TObject);
var
UserName: String; // User's login (text)
UserPassword: String; // User's password (text)
UserPasswordRAW: RawByteString; // User's password (bytes)
Salt: TSalt; // Salt (bytes)
SaltStr: String; // Salt (text)
Hash: TSHA256Hash; // Password's hash (bytes)
Hash2: TSHA256Hash; // Hash from database (bytes)
HashStr: String; // Password's hash (text)
begin
// Step 1. Create a new account
// Obtain user's credentials somehow:
UserName := InputBox('Sign in', 'Enter the login:', '');
UserPassword := InputBox('Sign in', 'Enter the password:', '');
// Create random bytes to be used as salt
Salt := InitSalt;
// Convert password (text) to bytes
UserPasswordRAW := UTF8Encode(UserPassword);
// Calculate hash from password (bytes) and salt (bytes) via HMAC
Hash := SHA256HMAC(@Salt, SizeOf(Salt), Pointer(UserPasswordRAW), Length(UserPasswordRAW));
// Convert bytes to text
SaltStr := HexEncodeToString(@Salt, SizeOf(Salt));
HashStr := HexEncodeToString(@Hash, SizeOf(Hash));
// Insert a new record into database
// This is a pseudo-code
InsertIntoDBTable('users', ['login', 'salt', 'password'], [UserName, SaltStr, HashStr]);
// Here:
// 'users' - table's name
// 'login' - string field of arbitrary length
// 'salt' - string field of 32 characters or binary field of 16 bytes
// 'password' - string field of 64 characters or binary field of 32 bytes
// Step 2. Authenticate a user
// Obtain user's credentials somehow:
UserName := InputBox('Log in', 'Enter the login:', '');
UserPassword := InputBox('Log in', 'Enter the password:', '');
// Search database for a user with the provided login
// This is a pseudo-code
// Real code should use "arguments"
Query := Format('SELECT salt, password FROM users WHERE login = ''%s'' LIMIT 1', [UserName]);
Values := DBQuery(Query);
// If there is no DB entry - then the login is not correct
if Length(Values) = 0 then
begin
ShowMessage('Invalid login');
Exit;
end;
// Convert salt and hash from text to bytes
SaltStr := Values[0]; // 'salt' field from SELECT
HashStr := Values[1]; // 'password' field from SELECT
Assert(HexCalcDecodedSize(Length(SaltStr)) = SizeOf(Salt));
HexDecodeFromString(SaltStr, @Salt);
Assert(HexCalcDecodedSize(Length(HashStr)) = SizeOf(Hash2));
HexDecodeFromString(HashStr, @Hash2);
// Calculate hash in the same way as above
UserPasswordRAW := UTF8Encode(UserPassword);
Hash := SHA256HMAC(@Salt, SizeOf(Salt), Pointer(UserPasswordRAW), Length(UserPasswordRAW));
// Now we have:
// Hash - hash (bytes) from the entered password
// Hash2 - hash (bytes) from the database
// If both hashes are equal - then password is correct
// Ensure correct password by comparing hashes
if CompareMem(@Hash, @Hash2, SizeOf(Hash)) then
ShowMessage('OK')
else
ShowMessage('Invalid password');
end;</pre>
<pre class="brush:php"><?php
// Step 1. Create a new account
// Obtain user's credentials somehow:
$UserName = $_GET['login'];
$UserPassword = $_GET['password'];
// Create random bytes to be used as salt
$Salt = random_bytes(16);
// Calculate hash from password (UTF-8 encoded) and salt (raw bytes) via HMAC
$HashStr = hash_hmac('sha256', $Salt, $UserPassword);
// Convert bytes to text
$SaltStr = bin2hex($Salt);
// Insert a new record into database
// This is a pseudo-code
InsertIntoDBTable('users', ['login', 'salt', 'password'], [$UserName, $SaltStr, $HashStr]);
// Here:
// 'users' - table's name
// 'login' - string field of arbitrary length
// 'salt' - string field of 32 characters or binary field of 16 bytes
// 'password' - string field of 64 characters or binary field of 32 bytes
// Step 2. Authenticate a user
// Obtain user's credentials somehow:
$UserName = $_GET['login'];
$UserPassword = $_GET['password'];
// Search database for a user with the provided login
// This is a pseudo-code
// Real code should use "arguments"
$Query = 'SELECT salt, password FROM users WHERE login = \'' . $UserName . '\' LIMIT 1';
$Values = DBQuery($Query);
// If there is no DB entry - then the login is not correct
if (empty($Values)) {
echo('Invalid login');
die;
}
// Convert salt and hash from text to bytes
$SaltStr = Values['salt']; // 'salt' field from SELECT
$HashStr2 = Values['password']; // 'password' field from SELECT
$Salt = hex2bin($SaltStr);
// Calculate hash in the same way as above
$HashStr = hash_hmac('sha256', $Salt, $UserPassword);
// Now we have:
// $HashStr - hash (text) from the entered password
// $HashStr2 - hash (text) from the database
// If both hashes are equal - then password is correct
// Ensure correct password by comparing hashes (as text)
if (strtolower($HashStr) == strtolower($HashStr2)) {
// or you can do this starting with PHP 5.6:
// if (hash_equals($HashStr2, $HashStr)) {
echo('OK'); // should get there,
// e.g. password is correct
} else {
echo('Invalid password');
}</pre>
<br />
<br />
<a name="t3"></a><h1>Encryption</h1>
EurekaLog has the following encryption functions:
<ul>
<li>In-process encryption (for example, to protect passwords)</li>
<li>Cross-process encryption (for example, to protect stored data within user account, or even within whole PC)</li>
<li><a href="https://en.wikipedia.org/wiki/Tiny_Encryption_Algorithm" title="Wikipedia: Tiny Encryption Algorithm">TEA</a></li>
<li><a href="https://en.wikipedia.org/wiki/Twofish" title="Wikipedia: Twofish">Twofish</a></li>
<li><a href="https://en.wikipedia.org/wiki/RSA_(cryptosystem)" title="Wikipedia: RSA (cryptosystem)">RSA</a></li>
</ul>
<br />
Similar to hash functions, encryption functions also accepts pointer+size or <code>RawByteString</code>. However, encryption functions have to <i>return</i> data of arbitrary size too. That is why you can also use the <a href="https://www.eurekalog.com/help/eurekalog/topic_type_eencrypt_tencryptbuffer.php" title="EurekaLog: TEncryptBuffer Record"><code>TEncryptBuffer</code></a> record, which just combines pointer and its size into a single argument.<br />
<br />
<h2>In-process encryption</h2>
Sometimes an application has to operate on "secret" information (such as user's password). It is necessary to store such sensitive information in an encrypted form to reduce the leakage risks. You can read more about this practice in <a href="https://docs.microsoft.com/en-us/windows/win32/secbp/handling-passwords" title="MSDN: Handling Passwords">MSDN</a> or (highly recommended) <a href="https://www.amazon.com/RSA-Securitys-Official-Guide-Cryptography/dp/007213139X" title="Amazon: RSA Security's Official Guide to Cryptography">book</a>. EurekaLog offers the following functions to protect sensitive information within a process:<br />
<ul>
<li><a href="https://www.eurekalog.com/help/eurekalog/topic_function_eencrypt_memprotect.php" title="EurekaLog: MemProtect Function"><code>MemProtect</code></a></li>
<li><a href="https://www.eurekalog.com/help/eurekalog/topic_function_eencrypt_memunprotect.php" title="EurekaLog: MemUnprotect Function"><code>MemUnprotect</code></a></li>
<li><a href="https://www.eurekalog.com/help/eurekalog/topic_function_eencrypt_securefree.php" title="EurekaLog: SecureFree Function"><code>SecureFree</code></a></li>
</ul>
The <code>MemProtect</code> function encrypts the specified memory block within a process in a such a way that it can be decrypted back only from the same process. The <code>MemUnprotect</code> function decrypts info back. The <code>SecureFree</code> function could be used to securely dispose of almost anything. This function will wipe (erase) memory before freeing it.<br />
<br />
For example:<br />
<pre class="brush:delphi">uses
EEncrypt; // for MemProtect/MemUnprotect and SecureFree
procedure TForm1.Button1Click(Sender: TObject);
var
UserPassword: String;
StoredPassword: TEncryptBuffer;
ClearText: TEncryptBuffer;
begin
// Prepare all buffers
FillChar(StoredPassword, SizeOf(StoredPassword), 0);
FillChar(ClearText, SizeOf(ClearText), 0);
// Obtain some confidential info
UserPassword := InputBox('Query', 'Enter the password:', '');
try
// Encrypt info immediately
ClearText.pbData := Pointer(UserPassword);
ClearText.cbData := Length(UserPassword) * SizeOf(Char);
MemProtect(ClearText, StoredPassword);
finally
// Wipe the confidential info
SecureFree(UserPassword);
// No need to dispose ClearText,
// because we did not allocate memory for it
end;
// ...
// Now we have StoredPassword - encrypted confidential info
// We would need to decrypt it each time we want to use it
// Don't forget to wipe unencrypted info once you have finished using it
// ...
// Decrypt the info:
MemUnprotect(StoredPassword, ClearText);
try
// Use confidential info somehow
Hash := MD5Hash(ClearText.pbData, ClearText.cbData);
finally
// Wipe unencrypted info after use
SecureFree(ClearText);
// You can also wipe derived info
SecureFree(Hash);
end;
// ...
// Clear (encrypted) confidential info when you no longer need it
SecureFree(StoredPassword);
end;</pre>
<br />
<h2>Cross-process encryption</h2>
Sometimes confidential information has to be stored somewhere. For example, a "Remember me" feature could write login/password pair into registry. The <code>MemProtect</code>/<code>MemUnprotect</code> function will not help you in such cases, because those functions will not work between/across processes (e.g. app's restart means creating a new process). Therefore, EurekaLog offers similar functions: <a href="https://www.eurekalog.com/help/eurekalog/topic_function_eencrypt_dataprotect.php" title="EurekaLog: DataProtect Function"><code>DataProtect</code></a> and <a href="https://www.eurekalog.com/help/eurekalog/topic_function_eencrypt_dataunprotect.php" title="EurekaLog: DataUnprotect Function"><code>DataUnprotect</code></a>. For example:<br />
<pre class="brush:delphi">uses
EEncrypt, // for DataProtect/DataUnprotect and SecureFree
EConfig, // for RegKeyWrite/RegKeyRead
EEncoding; // for Base64EncodeString/Base64DecodeString
procedure TForm1.Button1Click(Sender: TObject);
var
UserPassword: String;
StoredPassword: RawByteString;
ClearText: RawByteString;
begin
// Obtain some confidential info
UserPassword := InputBox('Query', 'Enter the password:', '');
try
// Convert to RawByteString for convenience
ClearText := UTF8Encode(UserPassword);
// Wipe the original form
SecureFree(UserPassword);
// Encrypt/protect the sensitive info
StoredPassword := DataProtect(ClearText);
// or:
// StoredPassword := DataProtect(ClearText, True);
// if you want to use HKEY_LOCAL_MACHINE below
// Wipe the confidential info
SecureFree(ClearText);
// Convert encrypted bytes to text
UserPassword := Base64EncodeString(StoredPassword);
// Store encrypted confidential info into registry
RegKeyWrite(HKEY_CURRENT_USER, '\SOFTWARE\YourApp', 'SavedPassword', UserPassword);
// Optional
SecureFree(UserPassword);
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(UserPassword);
SecureFree(StoredPassword);
SecureFree(ClearText);
end;
// ...
// Now we have encrypted sensitive info stored in the registry
// We need to read and decrypt it each time we want to use it
// Don't forget to wipe it after usage
// ...
// Read (encrypted) confidential info from the registry
UserPassword := RegKeyRead(HKEY_CURRENT_USER, '\SOFTWARE\YourApp', 'SavedPassword', '');
try
// Convert text back to bytes
StoredPassword := Base64DecodeString(UserPassword);
// Optional
SecureFree(UserPassword);
// Decrypt confidential info
ClearText := DataUnprotect(StoredPassword);
// Optional
SecureFree(StoredPassword);
// Need to convert back to text
UserPassword := UTF8ToString(ClearText);
// Wipe the unsecured data
SecureFree(ClearText);
// Use sensitive info somehow
Hash := MD5Hash(Pointer(UserPassword), Length(UserPassword) * SizeOf(Char));
// Wipe once finished
SecureFree(UserPassword);
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(UserPassword);
SecureFree(StoredPassword);
SecureFree(ClearText);
end;
end;</pre>
<br />
<h2>Symmetric encryption</h2>
EurekaLog supports <a href="https://ru.wikipedia.org/wiki/TEA" title="Википедия: TEA">TEA</a> and <a href="https://ru.wikipedia.org/wiki/Twofish" title="Википедия: Twofish">Twofish</a> symmetric ciphers. Both ciphers are not patented and can be used in any app. TEA is used in a wide variety of hardware due to its extremely low memory requirements and ease of implementation. Twofish is a robust general purpose symmetric encryption algorithm.<br />
<br />
Note that encrypted data can be larger than the original data - since symmetric encryption algorithms often operate in blocks of data ("chunks"). A distinctive feature of TEA is that the encrypted data will be equal in size to the original data, so the TEA encryption/decryption functions have an overloaded option for in-place operations, i.e. without memory reallocation. For the Twofish algorithm, the data size must be a multiple of the block size (16 bytes) - otherwise the data will be padded with the PKCS#5 algorithm to a minimum required size.<br />
<br />
EurekaLog offers <i>alg</i>Encrypt/<i>alg</i>Decrypt functions for both ciphers. All functions accepts key and source data. The difference is that Twofish functions allow you to additionally specify an optional <a href="https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Initialization_vector_(IV)" title="Wikipedia: Initialization vector (IV)">IV (initialization vector)</a>. Initialization vector is just a set of random bytes (which you can create by calling the <a href="https://www.eurekalog.com/help/eurekalog/topic_function_eencrypt_twofishinitiv.php" title="EurekaLog: TwofishInitIV Function"><code>TwofishInitIV</code> function</a>), which ensures that two equal source data will look different after encryption, e.g. it is something like <a href="https://en.wikipedia.org/wiki/Salt_(cryptography)" title="Wikipedia: Salt (cryptography)">salt</a> for hash. You don't need to store initialization vector securely, when used - it should be stored/transmitted together with encrypted data.<br />
<br />
Both cipers use binary keys (e.g. bytes) for encryption. Because cipers are "symmetric", this means that decryption key must be the same as encryption key. Obviosly, TEA key and Twofish key have different (but fixed) size. As a rule, encryption keys are not random, but are obtained from passwords entered by a user. To convert an arbitrary password into a key of a fixed length, the <i>alg</i>DeriveKey functions are used, which accepts a block of data of an arbitrary size. Key is derived from a password by a simple call to the hash function with an appropriate size result. For example, for TEA it will be MD5, for Twofish it will be SHA-256. The derive functions also have an overload that accepts the password as a string with an optional <a href="https://en.wikipedia.org/wiki/Salt_(cryptography)" title="Wikipedia: Salt (cryptography)">salt</a>. In this case, the password is converted to UTF-8 representation, and the hash is taken from the string 'salt' + 'UTF-8 password'. In addition, there is another version of derive functions: <i>alg</i>DeriveKeyHMAC function, which uses the HMAC algorithm to combine salt and password. In general, if you plan to use salt - we recommend using the <i>alg</i>DeriveKeyHMAC functions.<br />
<br />
If the encryption keys are obtained in some other way (e.g. not from a password) - you can also exchange keys directly, without "deriving" them from passwords. Just treat the key as a fixed size record/bytes array. The only pitfall is that EurekaLog uses optimization with Twofish: the key is not used directly, but is first converted into an intermediate version, which allows optimizing encryption and decryption operations. The original key is named <code>TTwofishRAWKey</code> and the optimized version is <code>TTwofishKey</code>.<br />
<br />
For example:<br />
<br />
<b>1. Encryption/decryption using a password:</b><br />
<pre class="brush:delphi">uses
EEncrypt; // for InitSalt, TEADeriveKey, TEAEncrypt/TEADecrypt, SecureFree
procedure TForm1.Button1Click(Sender: TObject);
var
Salt: TSalt; // Salt (random bytes)
Key: TTEAKey; // Key (derived from password)
Source: String; // Source text
SourceBytes: RawByteString; // Source text (bytes)
EncryptedBytes: RawByteString; // Encrypted text
begin
// Prepare all buffers
FillChar(Salt, SizeOf(Salt), 0);
FillChar(Key, SizeOf(Key), 0);
// Step 1: Encryption
try
// Define source data
Source := 'Привет';
// Define exact representation as bytes
// We use UTF-8 in this example
SourceBytes := UTF8Encode(Source);
// No need source data anywore - wipe it
SecureFree(Source);
// Generate random bytes to be used as salt
Salt := InitSalt;
// Derive key from password and salt
Key := TEADeriveKeyHMAC('a super secret password', Salt);
// Encrypt source data
EncryptedBytes := TEAEncrypt(Key, SourceBytes);
// Wipe everything that we no longer need
SecureFree(Key);
SecureFree(SourceBytes);
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(Source);
SecureFree(SourceBytes);
SecureFree(Key);
end;
// How we have:
// Salt - salt to derive key from password
// EncryptedBytes - encrypted data (arbitrary size)
// Both of those should be transferred to decryption side
// Step 2: Decryption
try
// Derive key from password and salt (salt should be passed to us)
Key := TEADeriveKeyHMAC('a super secret password', Salt);
// Now Key must match encryption key exactly
// Decrypt encrypted data
SourceBytes := TEADecrypt(Key, EncryptedBytes);
// Wipe key
SecureFree(Key);
// Optional
SecureFree(EncryptedBytes);
// Convert data back to text
Source := UTF8ToString(SourceBytes);
// Wipe unneeded data
SecureFree(SourceBytes);
// Use source data somehow
ShowMessage(Source);
// Wipe source data after usage
SecureFree(Source);
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(Source);
SecureFree(SourceBytes);
SecureFree(EncryptedBytes);
SecureFree(Key);
SecureFree(Salt);
end;
end;</pre>
<br />
<b>2. Exchange encrypted data between Delphi and PHP:</b><br />
<pre class="brush:delphi">uses
EEncrypt, // for all Twofish functions and SecureFree
EEncoding, // for Base64EncodeToString
ECore; // for ShellExec
procedure TForm1.Button1Click(Sender: TObject);
const
// Secret key known to both parties
// It must match key in PHP-script exactly
// Basically, this is just random bytes, which you can generate via TwofishInitSessionKeyRAW
// Obviosly, if you are going to use this example - you MUST replace this constant
// This is just an example. A real app may store this key somewhere
SecretKey: TTwofishRAWKey =
(160, 22, 228, 9, 73, 192, 173, 149,
154, 19, 115, 215, 74, 36, 20, 202,
178, 26, 103 , 47, 51, 4, 144, 20,
73, 153, 49, 160, 192, 25, 20, 114);
var
Key: TTwofishKey; // Optimized key (created from constant above)
IV: TTwofishInitVector; // Initialization vector (bytes)
Text: String; // Source data (text)
TextRAW: RawByteString; // Source data (bytes)
EncryptedText: RawByteString; // Encrypted data (bytes)
EncodedIV: String; // Initialization vector (text)
EncodedText: String; // Encrypted data (text)
URL: String; // URL to call PHP-script
ReplyRAW: RawByteString; // Reply from PHP-script (bytes)
Reply: String; // Reply from PHP-script (text)
begin
// Prepare buffers
FillChar(Key, SizeOf(Key), 0);
FillChar(IV, SizeOf(IV), 0);
try
// Obtain source data somehow
Text := 'Привет!';
// Convert text to bytes. Use UTF-8 in this example
TextRAW := UTF8Encode(Text);
// No longer needed
SecureFree(Text);
// Prepare (optimize) the key
Key := TwofishInitKey(SecretKey);
// Generate random bytes to be used as initialization vector
IV := TwofishInitIV;
// Encrypt source data
// This will use CBC mode, because initialization vector is used
EncryptedText := TwofishEncrypt(Key, TextRAW, @IV);
// Wipe source text
SecureFree(TextRAW);
// Convert bytes to text
EncodedIV := Base64EncodeToString(@IV, SizeOf(IV));
EncodedText := Base64EncodeString(EncryptedText);
// Optional
SecureFree(EncryptedText);
// Crearte URL to call PHP-script
// URLEncode is required to escape the '+' character
// If you are going to use HEX-encoding instead of Base64 - then URLEncode is not required
URL := Format('http://localhost/test.php?iv=%s&text=%s', [URLEncode(EncodedIV), URLEncode(EncodedText)]);
// Optional
SecureFree(EncodedIV);
SecureFree(EncodedText);
// Call PHP-script
if not InitWebTools then
RaiseLastOSError;
try
ReplyRAW := InternetGet(URL, [], []);
finally
DoneWebTools;
end;
// PHP-script did not returned anything?
if ReplyRAW = '' then
Abort;
// Optional
SecureFree(URL);
// Convert text to bytes
ReplyRAW := Base64DecodeString(Trim(String(ReplyRAW)));
// Decrypt data
TextRAW := TwofishDecrypt(Key, ReplyRAW, @IV);
// No longer needed
SecureFree(Key);
SecureFree(IV);
// Optional
SecureFree(ReplyRAW);
// Convert bytes back to text
Text := UTF8ToString(TextRAW);
// No longer needed
SecureFree(TextRAW);
// Use reply from PHP-script
ShowMessage(Text);
// Should show:
// 'Hello from PHP: Привет'
// Wipe the reply after use
SecureFree(Text);
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(Text);
SecureFree(TextRAW);
SecureFree(Key);
SecureFree(IV);
SecureFree(EncryptedText);
SecureFree(EncodedIV);
SecureFree(EncodedText);
SecureFree(URL);
SecureFree(ReplyRAW);
SecureFree(Reply);
end;
end;</pre>
<pre class="brush:php"><?php
// The functions below are required, because MCrypt use zero-padding instead of PKCS#5
// While OpenSSL does support PKCS#5, but it does not support Twofish
// PKCS#5 padding
function pkcs5_pad($text, $blocksize = 16) {
$pad = $blocksize - (strlen($text) % $blocksize);
return $text . str_repeat(chr($pad), $pad);
}
// PKCS#5 removal (unpadding)
function pkcs5_unpad($text) {
$pad = ord($text{strlen($text)-1});
if ($pad > strlen($text)) {
return false;
}
if (strspn($text, chr($pad), strlen($text) - $pad) != $pad) {
return false;
}
return substr($text, 0, -1 * $pad);
}
// Secret key known to both parties
// It must match key in Delphi exactly
// Basically, this is just random bytes, which you can generate via random_bytes(32)
// Obviosly, if you are going to use this example - you MUST replace this constant
// This is just an example. A real app may store this key somewhere
$Key = pack('C*',
160, 22, 228, 9, 73, 192, 173, 149,
154, 19, 115, 215, 74, 36, 20, 202,
178, 26, 103 , 47, 51, 4, 144, 20,
73, 153, 49, 160, 192, 25, 20, 114);
// Read passed data (initialization vector and encrypted data)
$EncodedIV = $_GET['iv'];
$EncodedText = $_GET['text'];
// Convert text to bytes
$IV = base64_decode($EncodedIV);
$EncryptedText = base64_decode($EncodedText);
// Decrypt data (you must use MCRYPT_TWOFISH256 instead of MCRYPT_TWOFISH in some versions of PHP)
$Text = mcrypt_decrypt(MCRYPT_TWOFISH, $Key, $EncryptedText, MCRYPT_MODE_CBC, $IV);
// Trim data to its real size
$Text = pkcs5_unpad($Text);
// Do something with source data from Delphi
$Text = 'Hello from PHP: ' . $Text;
// Pad data up to chunk border
$Text = pkcs5_pad($Text);
// Encrypt source data (you must use MCRYPT_TWOFISH256 instead of MCRYPT_TWOFISH in some versions of PHP)
$EncryptedText = mcrypt_encrypt(MCRYPT_TWOFISH, $Key, $Text, MCRYPT_MODE_CBC, $IV);
// Convert bytes to text
$EncodedText = base64_encode($EncryptedText);
// Send encrypted data back to Delphi
echo($EncodedText);</pre>
<br />
<b>3. Storing file encrypted:</b><br />
<pre class="brush:delphi">uses
EEncrypt, // for all Twofish functions and SecureFree
EEncoding, // for HexEncodeToString/HexDecodeString
EConfig, // for RegKeyWrite/RegKeyRead
ECompatibility; // for FileToString/StringToFile
procedure TForm1.Button1Click(Sender: TObject);
var
RAWKey: TTwofishRAWKey; // Encryption/decryption (session) key (RAW bytes)
Key: TTwofishKey; // Encryption/decryption (session) key (optimized)
IV: TTwofishInitVector; // Initialization vector
Content: RawByteString; // Source file (bytes)
EncryptedData: RawByteString; // Encrypted file
DataClear: RawByteString; // To encrypt encryption/session key
DataEncrypted: RawByteString; // To encrypt encryption/session key
DataStr: String; // Encrypted encryption/session key (text)
begin
// Prepare all buffers
FillChar(RAWKey, SizeOf(RAWKey), 0);
FillChar(Key, SizeOf(Key), 0);
FillChar(IV, SizeOf(IV), 0);
// Step 1: encrypt file on disk with random key
try
// Create random key (random bytes)
RAWKey := TwofishInitSessionKeyRAW;
// Optimize it for further use
Key := TwofishInitKey(RAWKey);
// Load whole file into a string
Content := FileToString(ParamStr(0));
// Content will be like 'MZP'#0#2#0#0#0...
// Use random bytes as initialization vector
IV := TwofishInitIV;
// Encrypt data (e.g. file)
EncryptedData := TwofishEncrypt(Key, Content, @IV);
// Wipe not needed data
SecureFree(Content);
SecureFree(Key);
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(Content);
SecureFree(Key);
end;
// Now we have:
// RAWKey - (secret) session key, which was used to encrypt the file
// IV - (open) initialization vector
// EncryptedData - encrypted file
// Step 2: protect session key
try
// Convert to RawByteString for convenience
DataClear := RAWToString(@RAWKey, SizeOf(RAWKey));
// Wipe key (no longer needed)
SecureFree(RAWKey);
// Encrypt encryption key
DataEncrypted := DataProtect(DataClear);
// Wipe unneccessary data
SecureFree(DataClear);
// Convert bytes to text
DataStr := HexEncodeToString(Pointer(DataEncrypted), Length(DataEncrypted));
// Optional
SecureFree(DataEncrypted);
// Store encrypted session key into registry
// We can safely do that, because this key can be decrypted back by the same user account only
RegKeyWrite(HKEY_CURRENT_USER, '\SOFTWARE\YourApp', 'EncryptionKey', DataStr);
// Optional
SecureFree(DataStr);
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(RAWKey);
SecureFree(DataClear);
SecureFree(DataEncrypted);
SecureFree(DataStr);
end;
// Now we have:
// Encrypted (secret) session (encryption) key in the registry
// IV - (open) initialization vector
// EncryptedData - encrypted file
// So, if we want to save encrypted file to disk - we need to save both IV and EncryptedData
// Only current user account would be able to decrypt data back
// Step 3: decrypt session key
try
// Read stored session key (as text)
DataStr := RegKeyRead(HKEY_CURRENT_USER, '\SOFTWARE\YourApp', 'EncryptionKey', '');
// Convert text to bytes
DataEncrypted := HexDecodeString(DataStr);
// Optional
SecureFree(DataStr);
// Decrypt session key
DataClear := DataUnprotect(DataEncrypted);
// Optional
SecureFree(DataEncrypted);
// Prepare session key (bytes)
Assert(Length(DataClear) = SizeOf(RAWKey));
RAWFromString(DataClear, @RAWKey);
// Wipe it's copy
SecureFree(DataClear);
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(DataStr);
SecureFree(DataEncrypted);
SecureFree(DataClear);
end;
// Now we have:
// RAWKey - (secret) key, which was used to encrypt the file
// IV - (open) initialization vector
// EncryptedData - encrypted file
// Step 4: decrypt the file
try
// Optimize the key for usage
Key := TwofishInitKey(RAWKey);
// Wipe source (no longer needed)
SecureFree(RAWKey);
// Decrypt the data (file)
// Both Key and IV must be exactly the same as in encryption at step 1 above
Content := TwofishDecrypt(Key, EncryptedData, @IV);
// Wipe the key (no longer needed)
SecureFree(Key);
// Optional
SecureFree(EncryptedData);
// Store unencrypted data back to file
StringToFile(ParamStr(0) + '.copy', Content);
// Now the Project1.exe.copy file must be exact copy of Project1.exe
// Wipe unnecessary data
SecureFree(Content);
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(RAWKey);
SecureFree(Key);
SecureFree(EncryptedData);
SecureFree(Content);
end;
end;</pre>
<br />
<h2>Asymmetric Cryptography</h2>
For asymmetric encryption - EurekaLog supports <a href="https://en.wikipedia.org/wiki/RSA_(cryptosystem)" title="Wikipedia: RSA (cryptosystem)">RSA</a>. The algorithm is not proprietary and can be freely used in any app. The asymmetry of the algorithm means that it uses two <i>different</i> keys: one key is used for encryption, the other key is used for decryption. One of the keys is kept secret, it is called "secret" or "private" key, and the other can be published - it is called "public" or "open" key. Whatever is encrypted with the public key can only be decrypted back with the matched private key and vice versa.<br />
<br />
<h3>Key Management</h3>
EurekaLog stores RSA keys into the <a href="https://www.eurekalog.com/help/eurekalog/topic_type_eencrypt_trsakey.php" title="EurekaLog: TRSAKey Type"><code>TRSAKey</code> record</a>, which have two fields: <a href="https://www.eurekalog.com/help/eurekalog/topic_field_eencrypt_trsakey_publickey.php" title="EurekaLog: TRSAKey.PublicKey Field"><code>PublicKey</code></a> to store public key and <a href="https://www.eurekalog.com/help/eurekalog/topic_field_eencrypt_trsakey_privatekey.php" title="EurekaLog: TRSAKey.PrivateKey Field"><code>PrivateKey</code></a> to store private key. You can create a brand new pair of keys by calling the <a href="https://www.eurekalog.com/help/eurekalog/topic_function_eencrypt_rsagenkey.php" title="EurekaLog: RSAGenKey Function"><code>RSAGenKey</code> function</a> (which will take a while - say, 5-15 seconds). As a rule, keys are not generated in apps, but ready-made (pre-generated) keys are loaded. EurekaLog offers the <code>RSA<i>Load/SavePublic/Private</i>Key</code> functions to load and save keys, for example: <a href="https://www.eurekalog.com/help/eurekalog/topic_function_eencrypt_rsaloadprivatekey.php" title="EurekaLog: RSALoadPrivateKey Function"><code>RSALoadPrivateKey</code></a>. EurekaLog supports few formats to export/import the keys, which are described by the <a href="https://www.eurekalog.com/help/eurekalog/topic_type_eencrypt_trsaexport.php" title="EurekaLog: TRSAExport Type"><code>TRSAExport/TRSAImport</code> type</a>:
<ul>
<li><code>rsBLOB</code> - this is a binary representation of a key with a header. The <a href="https://docs.microsoft.com/openspecs/windows_protocols/ms-mqqb/ade9efde-3ec8-4e47-9ae9-34b64d8081bb" title="MSDN: PUBLICKEYBLOB Type"><code>PUBLICKEYBLOB/PRIVATEKEYBLOB</code> record</a> from Microsoft is used as the header. Little-Endian.</li>
<li><code>rsDER</code> - this is a binary representation of a key, encoded into ASN.1 container without any headers (so called PKCS#1), Big-Endian. If you attempt to load ASN.1 container with a header (PKCS#8) - it will be silently ignored. It matches the <a href="https://docs.microsoft.com/windows/win32/seccrypto/constants-for-cryptencodeobject-and-cryptdecodeobject" title="MSDN: Constants for CryptEncodeObject and CryptDecodeObject"><code>RSA_CSP_PUBLICKEYBLOB/PKCS_RSA_PRIVATE_KEY</code></a> in CryptoAPI. As a rule, this format is used to save into <code>.der</code> files.</li>
<li><code>rsPEM</code> - this is a textual representation of a key. Basically, it is the same <code>rsDER</code>, but encoded into Base64. As a rule, this format is used to save into <code>.pem</code> files. <code>.key</code>, <code>.cert</code>, <code>.cer</code>, or <code>.crt</code> are also used. PKCS#1 compliant files use headers like <code>-----BEGIN/END RSA PRIVATE KEY-----</code>, <code>-----BEGIN/END RSA PUBLIC KEY-----</code>, while PKCS#8 complient files use headers like <code>-----BEGIN/END PRIVATE KEY-----</code>, <code>-----BEGIN/END PUBLIC KEY-----</code>.</li>
</ul>
EurekaLog does not support encrypted PKCS#1/PKCS#8, nor PKCS#7 and PKCS#12. PKCS#8 is supported by EurekaLog, but only when importing (see below).<br />
<br />
For example, the very same key can be exported like the following:
<ul>
<li><code>rsBLOB</code> (the <code>PUBLICKEYBLOB/PRIVATEKEYBLOB</code> header + little-endian, key $25 $17 $B4 $A0 ... $96 $B9 $9C $E7 starts from byte $15/21 and ends at the file's end):<br />
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEPbBAQkAWVpCRy9JJfZQfshBen-Xyj-Fs9CEL1K7r80OuYcspArW5X8MzRJe2rEbFYaDmtxFLbe0RCVBAaJdYKqE9Qzhip9I70TecGGMAanc-7QMzpX9e7cLKfiR-XSXr16p9xnvDFzoK/s0/BLOB.png" title="View Image"><img alt="rsBLOB" border="0" data-original-height="688" data-original-width="788" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEPbBAQkAWVpCRy9JJfZQfshBen-Xyj-Fs9CEL1K7r80OuYcspArW5X8MzRJe2rEbFYaDmtxFLbe0RCVBAaJdYKqE9Qzhip9I70TecGGMAanc-7QMzpX9e7cLKfiR-XSXr16p9xnvDFzoK/s0/BLOB.png" width="500" /></a></div>
</li>
<li><code>rsDER</code> (ASN.1-container PKCS#1 + big-endian, e.g. the same key $E7 $9C $B9 $96 ... $A0 $B4 $17 $25 starts with byte 10 and ends at byte 6 from file's end):<br />
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhouIJ5QD-0btGnRUVULwkcc5E6-5mr5AJpnInOJkjpBvJdwfvctVUtreo_okWiL39ZjO2RBAgYu8d4j8dBrwNMEXLdn8XCYly3OikXDZusvWVenOC_HbqnRAYUyJ7zXjBAv6yLuibUY7kk/s0/DER.png" title="View Image"><img alt="rsPEM" border="0" data-original-height="672" data-original-width="784" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhouIJ5QD-0btGnRUVULwkcc5E6-5mr5AJpnInOJkjpBvJdwfvctVUtreo_okWiL39ZjO2RBAgYu8d4j8dBrwNMEXLdn8XCYly3OikXDZusvWVenOC_HbqnRAYUyJ7zXjBAv6yLuibUY7kk/s0/DER.png" width="500" /></a></div>
</li>
<li><code>rsPEM</code> (Base64-encoded ASN.1-container PKCS#1):<br />
<pre>-----BEGIN RSA PUBLIC KEY-----
MIICCgKCAgEA55y5ll1KryRC7umxntWX7t3zOP3qUVxQo7gin3sA1dePyzLxTxtE
47R+/sqkgFygXdlBqnmjbwu60kU2Zd7k7QFGhZWqfPcAYI3xd660vUPnmXK7n2R1
3AtF2BW/5MqIH7D3ddjLCt5CoUn6KRZSuz+pySDpuquKerRB5Gq/0WjUG2IIcQXU
Z1i4qMicPhbOJH76rFPgRngBuvJtS0UCBKx4YOlK0q1JUUJ1leSGp2gAjYGrD7fN
SOU8r70a97NDu4UblmsS9zW29OHAEF7jNFsVNVBU78P/XZ4hmL41gaPRGws3HXfA
vGbVattUzHTHsHMJeRLoiPAgak3TqAM2px7qOcNNN8FB91XbnxzvPARfDrBMbpc4
OcWmDSMuc1RGI/mQCIlGRvA2nhD7Dfu3L5sxnrjjOC+LLpIVsGe5+cs1ZkfD7kII
AzV/MXXNlx366n/Z1+u97VocmvHcqVCl/s9AMqdXflzAYD+9p7bXhJdP9XfOXf9z
zCyPBK/Iyk+B4lRR9cmuBW7FAq1JM3PZWZ2mEx0fgrL8M0w5cf2Ts84XtNIEFDa6
MFOe48sJfDIiPPw4ePohSuYpAY71Du2cQe87VQAf/caclWsrFplItilN93Xx5kQW
5S16HHLc7A+EKEaNBnUsNl+n0/99jjfHA9PAqxFVaVT68X9eSKC0FyUCAwEAAQ==
-----END RSA PUBLIC KEY-----</pre></li>
</ul>
<br />
<b>Creating and exporting a new key pair:</b><br />
<ol>
<li>Delphi (PKCS#1):<br />
<pre class="brush:delphi">uses
EEncrypt; // for all RSA functions
procedure TForm1.Button1Click(Sender: TObject);
var
Key: TRSAKey;
begin
// Create a new key pair (takes few seconds)
Key := RSAGenKey;
try
// Export both keys into all formats (only as an example)
RSASavePublicKey(Key, 'C:\Documents\public.blob', rsBLOB);
RSASavePublicKey(Key, 'C:\Documents\public.der', rsDER);
RSASavePublicKey(Key, 'C:\Documents\public.pem', rsPEM);
RSASavePrivateKey(Key, 'C:\Documents\private.blob', rsBLOB);
RSASavePrivateKey(Key, 'C:\Documents\private.der', rsDER);
RSASavePrivateKey(Key, 'C:\Documents\private.pem', rsPEM);
finally
// Wipe all keys
SecureFree(Key);
end;
end;</pre>
</li>
<li>PHP (PKCS#8):<br />
<pre class="brush:php"><?php
// Create a new key pair (takes few seconds)
$config = array(
"private_key_bits" => 4096,
"private_key_type" => OPENSSL_KEYTYPE_RSA,
"encrypt_key" => false
);
$privateKey = openssl_pkey_new($config);
$publicKey = openssl_pkey_get_public($privateKey);
// Save keys into files
openssl_pkey_export($privateKey, $PEM, null, $config);
file_put_contents('./private.pem', $PEM);
file_put_contents('./public.pem', $publicKey['key']);</pre>
</li>
<li>OpenSSL (PKCS#8):<br />
<pre>openssl genpkey -out private.pem -algorithm RSA -pkeyopt rsa_keygen_bits:4096
openssl rsa -in private.pem -out public.pem -outform pem -pubout</pre>
</li>
</ol>
<br />
If you want to exchange keys between Delphi/EurekaLog, Windows/WinCrypt, and PHP/OpenSSL - you need to use an ASN.1-container format (i.e. binary DER or text PEM). There is one pitfall here:
<ol>
<li>Windows/WinCrypt exports/imports only keys themselfes into ASN.1-container. Such format is called PKCS#1, you can distinguish it in a text form (PEM) by the comments like <code>-----BEGIN RSA PUBLIC KEY-----</code>. If you open such PEM/DER file into any ASN.1 editor/viewer/decoder (or use the <code>openssl asn1parse -in private.pem</code> command) - you should see something like this:
<pre>SEQUENCE (9 elem)
INTEGER 0
INTEGER (4096 bit) 758666102228921792910938751013886686183781000609266742264329283135491…
INTEGER 65537
INTEGER (4096 bit) 626038158727742962915964533848528190012037116635627896022932505790124…
INTEGER (2048 bit) 265173868639039499374763037151257212413358779720904507462546669893261…
INTEGER (2048 bit) 286101381754789259717271171537486143896554619163648315398099842455950…
INTEGER (2048 bit) 241273140164272477344356926702923044634459679795497746005945617372402…
INTEGER (2047 bit) 117977475958972484914769571551956345862709440207784850140129213152449…
INTEGER (2047 bit) 114103066753170488160676998988478007480145103422326665392933549037964…</pre>
8 feilds describe various components of the key (module, exponent, etc.).</li>
<li>PHP/OpenSSL exports not just keys, but also stores additional information, such as alg ID, version, etc. Such format is called PKCS#8, you can distinguish it in a text form (PEM) by the comments like <code>-----BEGIN PRIVATE KEY-----</code>. If you open such PEM/DER file into any ASN.1 editor/viewer/decoder - you should see something like this:
<pre>SEQUENCE (3 elem)
INTEGER 0
SEQUENCE (2 elem)
OBJECT IDENTIFIER 1.2.840.113549.1.1.1 rsaEncryption (PKCS #1)
NULL
OCTET STRING (2349 byte) 30820929020100028202010098D61F2FBA3EC958DB082F286781EE7CC258ADCE2B0A…
SEQUENCE (9 elem)
INTEGER 0
INTEGER (4096 bit) 758666102228921792910938751013886686183781000609266742264329283135491…
INTEGER 65537
INTEGER (4096 bit) 626038158727742962915964533848528190012037116635627896022932505790124…
INTEGER (2048 bit) 265173868639039499374763037151257212413358779720904507462546669893261…
INTEGER (2048 bit) 286101381754789259717271171537486143896554619163648315398099842455950…
INTEGER (2048 bit) 241273140164272477344356926702923044634459679795497746005945617372402…
INTEGER (2047 bit) 117977475958972484914769571551956345862709440207784850140129213152449…
INTEGER (2047 bit) 114103066753170488160676998988478007480145103422326665392933549037964…</pre>
The key is stored in the same way as in PKCS#1, but there is also an additional header added.</li>
</ol>
Different encryption libraries support various formats. If you attempt to pass key in an unsupported format - you would get errors like these:
<pre>CRYPT_E_ASN1_BADTAG (8009310B): ASN1 bad tag value met</pre>
<pre>openssl_pkey_get_private: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag</pre>
<br />
EurekaLog saves keys into PKCS#1, but it is able to load both PKCS#1 and PKCS#8. OpenSSL saves into PKCS#8, but it supports loading both PKCS#8 and PKCS#1.<br />
<br />
<blockquote><font color="red"><b>IMPORTANT</b></font><br />
OpenSSL supports PKCS#1/PKCS#8 PEM only when it has the proper comment! E.g. the PKCS#1 file <b>must</b> start with the <code>-----BEGIN RSA PRIVATE KEY-----</code> or <code>-----BEGIN RSA PUBLIC KEY-----</code>, while PKCS#8 file <b>must</b> start with the <code>-----BEGIN PRIVATE KEY-----</code> or <code>-----BEGIN PUBLIC KEY-----</code>.</blockquote>
<br />
P.S. You can also convert PKCS#1 (Windows/WinCrypt/EurekaLog) into PKCS#8 (PHP/OpenSSL) via:<br />
<pre>openssl pkcs8 -topk8 -inform pem -in private.pem -outform pem -nocrypt -out private2.pem
openssl rsa -RSAPublicKey_in -in public.pem -pubout -out public2.pem</pre>
and back:
<pre>openssl rsa -inform pem -in private.pem -outform pem -out private2.pem
openssl rsa -pubin -in public.pem -RSAPublicKey_out -out public2.pem</pre>
<br />
P.P.S. Latest versions of OpenSSL supports BLOB:<br />
Convert PEM to BLOB:<br />
<pre>openssl rsa -inform PEM -in private.pem -outform "MS PRIVATEKEYBLOB" -out private.blob</pre>
Convert BLOB to PEM:<br />
<pre>openssl rsa -inform "MS PRIVATEKEYBLOB" -in private.blob -outform PEM -out private.pem</pre>
Change key format to <code>"MS PUBLICKEYBLOB"</code> for a public key, or you can simply convert private key, then extract public key from the private key.<br />
<br />
In summary, we recommend:
<ol>
<li>Use the EurekaLog Crypto Helper tool to create a pair of new RSA keys:
<ol>
<li>Run Start / Programs / EurekaLog / Tools / EurekaLog Crypto Helper</li>
<li>Go to the Keys tab</li>
<li>Go to the RSA tab</li>
<li>Hit Create New button</li>
<li>Save private and public keys into <code>private.pem</code> and <code>public.pem</code> files. This will save keys into PKCS#1.</li>
</ol>
Alternatively you can use:
<pre class="brush:delphi">uses
EEncrypt;
procedure TForm1.Button1Click(Sender: TObject);
var
Key: TRSAKey;
begin
Key := RSAGenKey;
try
RSASavePublicKey (Key, 'C:\Documents\public.pem', rsPEM);
RSASavePrivateKey(Key, 'C:\Documents\private.pem', rsPEM);
finally
SecureFree(Key);
end;
end;</pre>
</li>
<li>Import keys into Delphi (EurekaLog supports PKCS#1):<br />
<pre class="brush:delphi">var
RSAKey: TRSAKey;
begin
// Loads PKCS#1 PEM
RSAKey := RSALoadPrivateKey('C:\Documents\private.pem', rsPEM);</pre>
<pre class="brush:delphi">var
RSAKey: TRSAKey;
begin
// Loads PKCS#1 PEM
RSAKey := RSALoadPublicKey('C:\Documents\public.pem', rsPEM);</pre></li>
<li>Import keys into PHP (OpenSSL from PHP supports PKCS#1):<br />
<pre class="brush:php"><?php
$PrivateKey = <<<EOD
-----BEGIN RSA PRIVATE KEY-----
MIIJKgIBAAKCAgEA5CBBmb8QlDVt7uqKZPW6I/GcjyzDg+7VuZd5OcQXVpglsWoa
...
aNz1gCLcqrQiTXHTVg821kYszBDySjfQGJQ3JJhf1/9XGcVjcopbWWeeNpHs5w==
-----END RSA PRIVATE KEY-----
EOD;
// Or:
// $PrivateKey = 'file:///var/www/private.pem';
$PrivateKey = openssl_pkey_get_private($PrivateKey);
if (!PrivateKey) {
echo('openssl_pkey_get_private: ' . openssl_error_string());
die();
}</pre>
<pre class="brush:php"><?php
$PublicKey = <<<EOD
-----BEGIN RSA PUBLIC KEY-----
MIICCgKCAgEA5CBBmb8QlDVt7uqKZPW6I/GcjyzDg+7VuZd5OcQXVpglsWoauYvq
...
f69nl8KyfHhsqffkDeDIaA73hspgFM5bh2zGdj4n8101bjHRu8N35qECAwEAAQ==
-----END RSA PUBLIC KEY-----
EOD;
// Or:
// $PublicKey = 'file:///var/www/public.pem';
$PublicKey = openssl_pkey_get_public($PublicKey);
if (!PublicKey) {
echo('openssl_pkey_get_public: ' . openssl_error_string());
die();
}</pre>
It is important that keys must start with a correct comment. E.g. <code>-----BEGIN RSA PRIVATE KEY-----</code> and <code>-----BEGIN RSA PUBLIC KEY-----</code>.
</li>
</ol>
Alternatively, you can:<br />
<ol>
<li><a href="https://wiki.openssl.org/index.php/Binaries" title="OpenSSL Binaries">Download OpenSSL for Windows</a>.</li>
<li>Create a new key pair. The commands below will create two files (<code>private.pem</code> and <code>public.pem</code>) as PKCS#8 PEM:<br />
<pre>openssl genpkey -out private.pem -algorithm RSA -pkeyopt rsa_keygen_bits:4096
openssl rsa -in private.pem -out public.pem -outform pem -pubout</pre></li>
<li>Import keys in Delphi (EurekaLog supports PKCS#8):<br />
<pre class="brush:delphi">var
RSAKey: TRSAKey;
begin
// Loads PKCS#8 PEM
RSAKey := RSALoadPrivateKey('C:\Documents\private.pem', rsPEM);</pre>
<pre class="brush:delphi">var
RSAKey: TRSAKey;
begin
// Loads PKCS#8 PEM
RSAKey := RSALoadPublicKey('C:\Documents\public.pem', rsPEM);</pre></li>
<li>Import keys in PHP (OpenSSL in PHP supports PKCS#8):<br />
<pre class="brush:php"><?php
$PrivateKey = <<<EOD
-----BEGIN PRIVATE KEY-----
MIIJRAIBADANBgkqhkiG9w0BAQEFAASCCS4wggkqAgEAAoICAQDkIEGZvxCUNW3u
...
X+VZcAA+CUNp1xYUuep0UoeqYtfzTuKvinNo3PWAItyqtCJNcdNWDzbWRizMEPJK
N9AYlDckmF/X/1cZxWNyiltZZ542kezn
-----END PRIVATE KEY-----
EOD;
// Or:
// $PrivateKey = 'file:///var/www/private.pem';
$PrivateKey = openssl_pkey_get_private($PrivateKey);
if (!PrivateKey) {
echo('openssl_pkey_get_private: ' . openssl_error_string());
die();
}</pre>
<pre class="brush:php"><?php
$PublicKey = <<<EOD
-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA5CBBmb8QlDVt7uqKZPW6
...
v4sIP6NMmNKN8TwtqUKxcjZJMrVhjPJWf69nl8KyfHhsqffkDeDIaA73hspgFM5b
h2zGdj4n8101bjHRu8N35qECAwEAAQ==
-----END PUBLIC KEY-----
EOD;
// Or:
// $PublicKey = 'file:///var/www/public.pem';
$PublicKey = openssl_pkey_get_public($PublicKey);
if (!PublicKey) {
echo('openssl_pkey_get_public: ' . openssl_error_string());
die();
}</pre>
It is important that keys must start with a correct comment. E.g. <code>-----BEGIN PRIVATE KEY-----</code> and <code>-----BEGIN PUBLIC KEY-----</code>.
</li>
</ol>
<br />
Note: the <code>openssl_pkey_get_private</code>/<code>openssl_pkey_get_public</code> calls are optional. If the key variable contains the key itself in PEM format, then it can be passed directly to OpenSSL functions. In this case, the <code>openssl_pkey_get_private</code>/<code>openssl_pkey_get_public</code> functions are called only as an example and to check that the key was specified correctly.<br />
<br />
<h3>Asymmetric Encryption</h3>
EurekaLog offers the <a href="https://www.eurekalog.com/help/eurekalog/topic_function_eencrypt_rsaencrypt.php" title="EurekaLog: RSAEncrypt Function"><code>RSAEncrypt</code></a> and <a href="https://www.eurekalog.com/help/eurekalog/topic_function_eencrypt_rsadecrypt.php" title="EurekaLog: RSADecrypt Function"><code>RSADecrypt</code></a> functions, which are used in a similar way to the symmetric encryption functions above. There are only a few differences:
<ol>
<li>Because asymmetric encryption uses two different keys, it is used in sender-recipient scenarios and is not used when the same person encrypts and decrypts data.</li>
<li>Since asymmetric encryption is very slow, it is never applied to the open data itself. Instead, the open data is encrypted with any symmetric cipher with a random key (called a "session key"), and then the symmetric session key is encrypted with asymmetric encryption.</li>
<li>The public key is used for encryption, and the private key is used for decryption. Therefore anyone can encrypt data with the recipient's public key, while only the recipient can decrypt the data. This is how secrecy is ensured.</li>
</ol>
<br />
<code>RSAEncrypt</code>/<code>RSADecrypt</code> functions work with little-endian data and use PKCS#1 Type 2 padding.<br />
<br />
<b>Encrypting file on a disk:</b><br />
<pre class="brush:delphi">uses
EEncrypt; // for all RSA functions and SecureFree
procedure TForm1.Button1Click(Sender: TObject);
var
// (Random) session key to encrypt the file
SessionKey: TTwofishKey;
SessionKeyRAW: TTwofishRAWKey;
// Asymmetric key to encrypt the session key
RSAKey: TRSAKey;
// Open data (file) to encrypt
Data: TMemoryStream;
// Encrypted data
EncryptedData: TEncryptBuffer;
// Stream to save the encrypted file
FS: TFileStream;
begin
// Prepare all buffers
FillChar(SessionKey, SizeOf(SessionKey), 0);
FillChar(SessionKeyRAW, SizeOf(SessionKeyRAW), 0);
FillChar(RSAKey, SizeOf(RSAKey), 0);
FillChar(EncryptedData, SizeOf(EncryptedData), 0);
try
// Create a new (random) session key
SessionKeyRAW := TwofishInitSessionKeyRAW;
SessionKey := TwofishInitKey(SessionKeyRAW);
// FS is used twice:
// 1). To store encrypted session key
// 2). To store encrypted data
FS := nil;
try
// Step 1: encrypt session key
// Load public key from a file (must be prepared beforehand)
RSAKey := RSALoadPublicKey('C:\Documents\public.pem', rsPEM);
try
// Encrypt session key (SessionKeyRAW) into EncryptedData buffer
EncryptedData.cbData := SizeOf(SessionKeyRAW);
RSAEncrypt(RSAKey, @SessionKeyRAW, Pointer(EncryptedData.pbData), EncryptedData.cbData);
finally
// No longer need public key
SecureFree(RSAKey);
// No longer need source for the session key
SecureFree(SessionKeyRAW);
end;
try
// Now: EncryptedData stored encrypted session key
// Save encrypted session key into destination file
FS := TFileStream.Create('C:\Documents\EncryptedData.bin', fmCreate or fmShareExclusive);
FS.WriteBuffer(EncryptedData.cbData, SizeOf(EncryptedData.cbData));
FS.WriteBuffer(EncryptedData.pbData^, EncryptedData.cbData);
// Don't close FS yet, we are not finished saving data...
finally
// Wipe encrypted session key
SecureFree(EncryptedData);
end;
// Now we have SessionKey and destination file FS
// Step 2: encrypt the file
Data := TMemoryStream.Create;
try
// Load entire file into memory
Data.LoadFromFile('C:\Documents\Text.txt');
// Encrypt the file
// We are not using initialization vector, so ECB mode will be used
EncryptedData.cbData := Cardinal(Data.Size);
TwofishEncrypt(SessionKey, Data.Memory, Pointer(EncryptedData.pbData), EncryptedData.cbData);
// No longer needed
SecureFree(SessionKey);
// Now save encrypted file into destination file
FS.WriteBuffer(EncryptedData.pbData^, EncryptedData.cbData);
finally
// Wipe everything
SecureFree(Data);
SecureFree(EncryptedData);
end;
finally
// Close the file
FreeAndNil(FS);
end;
// Now the C:\Documents\EncryptedData.bin file is ready
// It contains encrypted (random) session key, as well as encrypted C:\Documents\Text.txt file
// This file can be descrypted by someone, who has the private key
// E.g. you can safely pass this file to the recipient by any means, including unsecured channels
finally
// Wipe the remaining data
SecureFree(SessionKey);
end;
end;
procedure TForm1.Button2Click(Sender: TObject);
var
// (Random) session key to decrypt the file
SessionKey: TTwofishKey;
// Asymmetric key to decrypt the session key
RSAKey: TRSAKey;
// Encrypted file
Data: TMemoryStream;
// Decrypted file
DecryptedData: TEncryptBuffer;
// Stream to load the encrypted file
FS: TFileStream;
begin
// Prepare all buffers
FillChar(SessionKey, SizeOf(SessionKey), 0);
FillChar(RSAKey, SizeOf(RSAKey), 0);
FillChar(DecryptedData, SizeOf(DecryptedData), 0);
// Open encrypted file
FS := TFileStream.Create('C:\Documents\EncryptedData.bin', fmOpenRead or fmShareDenyWrite);
try
try
// Step 1: decrypt the session key
// Read encrypted session key from the file
FS.ReadBuffer(DecryptedData.cbData, SizeOf(DecryptedData.cbData));
DecryptedData.pbData := AllocMem(DecryptedData.cbData);
try
FS.ReadBuffer(DecryptedData.pbData^, DecryptedData.cbData);
// Decrypt symmetric session key with private asymmetric key
RSAKey := RSALoadPrivateKey('C:\Documents\private.pem', rsPEM);
try
EEncrypt.RSADecrypt(RSAKey, DecryptedData);
finally
SecureFree(RSAKey);
end;
// Initialize the session key
Assert(DecryptedData.cbData = SizeOf(TTwofishRAWKey));
SessionKey := TwofishInitKey(TTwofishRAWKey(Pointer(DecryptedData.pbData)^));
finally
// Wipe unneeded data
SecureFree(DecryptedData);
end;
// Now we have symmetric SessionKey, which we can use to decrypt the rest of the file
// Step 2: decrypt the file
Data := TMemoryStream.Create;
try
// The remaining data in the file represents the encrypted file
Data.CopyFrom(FS, FS.Size - FS.Position);
// Decrypt the file
DecryptedData.cbData := Cardinal(Data.Size);
TwofishDecrypt(SessionKey, Data.Memory, Pointer(DecryptedData.pbData), DecryptedData.cbData);
// No longer needed
SecureFree(SessionKey);
// Save decrypted file to disk
FreeAndNil(FS);
FS := TFileStream.Create('C:\Documents\Text2.txt', fmCreate or fmShareExclusive);
try
FS.WriteBuffer(DecryptedData.pbData^, DecryptedData.cbData);
finally
FreeAndNil(FS);
end;
// No longer needed
SecureFree(DecryptedData);
// Now Text2.txt should be exact copy of Text.txt
finally
SecureFree(Data);
end;
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(SessionKey);
SecureFree(DecryptedData);
end;
finally
FreeAndNil(FS);
end;
end;</pre>
<br />
<h3>Digital Signature</h3>
When a private key is used for encryption and a public key is used for decryption - it is called a "digital signature". Anyone can decrypt the data (since everyone has the public key), therefore secrecy of data is not ensured in this way. But on the other hand, if the data can be decrypted with someone's public key, we can be sure that the data was encrypted by him (because only this person has the secret private key). This way we can check authenticity of the data.<br />
<br />
However, as a rule, data itself is not encrypted by private key. A hash is calculated from the data and the hash is then encrypted instead. EurekaLog offers <a href="https://www.eurekalog.com/help/eurekalog/topic_function_eencrypt_rsasign.php" title="EurekaLog: RSASign Function"><code>RSASign</code></a> (uses private key) and <a href="https://www.eurekalog.com/help/eurekalog/topic_function_eencrypt_rsaverify.php" title="EurekaLog: RSAVerify Function"><code>RSAVerify</code></a> (uses public key) functions to sign and verify. EurekaLog's digital signature functions use SHA1 with EMSA-PKCS1 padding.<br />
<br />
The resulting digital signature is an opaque array of bytes of arbitrary length. If you want to exchange digital signatures with other environments, remember that Windows/Delphi use little-endian byte order, while some other environments (e.g. .NET or PHP) use big-endian. Therefore, in some cases, the byte order of a digital signature needs to be reversed.<br />
<br />
For example:<br />
<br />
<b>Requesting software license from a PHP-script:</b><br />
<pre class="brush:delphi">uses
EEncrypt, // for all RSA functions and SecureFree
EEncoding, // for Base64
EJSON, // for JSON functions
EWebTools; // for network functions
procedure TForm1.Button1Click(Sender: TObject);
var
// Data to be send to a PHP-script
JSON, JSONRequest, JSONUser: IJSONValues;
JSONText: String;
JSONRAW: RawByteString;
// (Random) symmetric key to encrypt the data
SessionKey: TTwofishKey;
SessionKeyRAW: TTwofishRAWKey;
// Asymmetric key to encrypt the session key
RSAKey: TRSAKey;
// Encrypted data (bytes)
EncryptedData: TEncryptBuffer;
// Encrypted data (text)
EncodedKey, EncodedData: String;
// URL to call PHP-script
URL: String;
// Reply from PHP (bytes)
ReplyRAW: RawByteString;
// License (text)
EncodedLicense: String;
// Digital signature (text)
EncodedSignature: String;
// License (bytes)
License: RawByteString;
// Digital signature (bytes)
Signature: RawByteString;
begin
// Prepare all buffers
FillChar(SessionKey, SizeOf(SessionKey), 0);
FillChar(SessionKeyRAW, SizeOf(SessionKeyRAW), 0);
FillChar(RSAKey, SizeOf(RSAKey), 0);
FillChar(EncryptedData, SizeOf(EncryptedData), 0);
try
// Step 1: prepare some data to send to a PHP-script
// The code below is just an example
JSON := JSONCreate;
JSONRequest := JSONCreate;
JSONUser := JSONCreate;
JSONRequest['version'] := 1;
JSONRequest['type'] := 'license';
JSONRequest['scope'] := 'installer';
JSONUser['login'] := 'input-from-edit1';
JSONUser['password'] := 'input-from-edit2';
JSON['app'] := 'MyApp';
JSON['version'] := GetModuleVersion(GetModuleName(HInstance));
JSON['date'] := Now;
JSON['request'] := JSONRequest;
JSON['user'] := JSONUser;
JSONText := JSON.ToString;
Finalize(JSONUser); // optional
Finalize(JSONRequest); // optional
Finalize(JSON); // optional
(*
Now JSONText contains:
{
"app": "MyApp",
"version": "1.0.0.0",
"date": "2021.06.25 14:04:21",
"request": {
"version": 1,
"type": "license",
"scope": "installer"
}
"user": {
"login": "input-from-edit1",
"password": "input-from-edit2"
}
}
*)
JSONRAW := UTF8Encode(JSONText);
SecureFree(JSONText);
// Now JSONRAW contains bytes to send to a PHP-script
// Step 2: encrypt request data and send encrypted data to a PHP-script
// Generate a new random key
SessionKeyRAW := TwofishInitSessionKeyRAW;
SessionKey := TwofishInitKey(SessionKeyRAW);
// Step 2a: encrypt the session key
// Load public key from a file
RSAKey := RSALoadPublicKey('C:\Documents\public.pem', rsPEM);
try
// Encrypt SessionKeyRAW into EncryptedData buffer
EncryptedData.cbData := SizeOf(SessionKeyRAW);
RSAEncrypt(RSAKey, @SessionKeyRAW, Pointer(EncryptedData.pbData), EncryptedData.cbData);
try
// No longer needed
SecureFree(RSAKey);
SecureFree(SessionKeyRAW);
// Convert bytes to text
EncodedKey := Base64EncodeToString(EncryptedData.pbData, EncryptedData.cbData);
// No longer needed
SecureFree(EncryptedData);
finally
// На всякий случай (исключение) - чистим данные
SecureFree(EncryptedData);
end;
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(RSAKey);
SecureFree(SessionKeyRAW);
end;
// Now:
// - EncodedKey stores encrypted session key (as a text)
// - SessionKey stores session key (ready to use)
// - JSONRAW stores request's bytes
// Step 2b: encrypt the request
// Since we do not use initialization vector - encryption will use ECB mode
EncryptedData.cbData := Length(JSONRAW);
TwofishEncrypt(SessionKey, Pointer(JSONRAW), Pointer(EncryptedData.pbData), EncryptedData.cbData);
try
// No longer needed
SecureFree(SessionKey);
// Convert bytes to text
EncodedData := Base64EncodeToString(EncryptedData.pbData, EncryptedData.cbData);
// No longer needed
SecureFree(EncryptedData);
// Send both encrypted data and session key to the PHP-script
// URLEncode is required to escape '+' in Base-64
URL := 'http://localhost/test.php?key=' + URLEncode(EncodedKey) + '&data=' + URLEncode(EncodedData);
if not InitWebTools then
RaiseLastOSError;
try
ReplyRAW := InternetGet(URL, [], []);
finally
DoneWebTools;
end;
// PHP-script does not return anything?
// (should not happen normally)
if ReplyRAW = '' then
Abort;
// Optional
SecureFree(URL);
// No longer needed
SecureFree(EncodedKey);
SecureFree(EncodedData);
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(SessionKey);
SecureFree(EncryptedData);
SecureFree(EncodedKey);
SecureFree(EncodedData);
end;
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(SessionKey);
SecureFree(SessionKeyRAW);
SecureFree(EncodedData);
SecureFree(EncodedKey);
SecureFree(JSONText);
SecureFree(JSONRAW);
end;
// Now we have ReplyRAW - bytes from the PHP-script
try
// Convert bytes to text
JSONText := UTF8ToString(ReplyRAW);
// Optional
Finalize(ReplyRAW);
// Convert JSON-text to JSON-object
JSON := JSONCreate(JSONText);
// Optional
Finalize(JSONText);
// PHP-script has returned some error?
if JSON.IndexOf('error') >= 0 then
raise Exception.Create(JSON['error']);
// Extract license and digital signature (text)
EncodedLicense := JSON['license'];
EncodedSignature := JSON['signature'];
// Convert text to bytes
License := Base64DecodeString(EncodedLicense);
Signature := Base64DecodeString(EncodedSignature);
// Load public key from a file
RSAKey := RSALoadPublicKey('C:\Documents\public.pem', rsPEM);
try
// Verify the digital signature
if RSAVerify(RSAKey, Pointer(License), Length(License), Pointer(Signature), Length(Signature)) then
begin
// Digital signature is not broken
// This means that License really came from our server
// Optional
SecureFree(RSAKey);
// Just as an example
// Real application would probably have License encrypted
ShowMessage(UTF8ToString(License));
// Will show 'This is just an example license'
end
else
ShowMessage('Not signed');
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(RSAKey);
end;
finally
// Just in case (e.g. exception) - wipe everything
SecureFree(JSONText);
SecureFree(ReplyRAW);
SecureFree(EncodedSignature);
SecureFree(EncodedLicense);
SecureFree(Signature);
SecureFree(License);
end;
end;</pre>
<pre class="brush:php"><?php
// These functions are required, because MCrypt uses zero padding instead of PKCS#5
// OpenSSL supports PKCS#5, but does not support Twofish
// PKCS#5 padding
function pkcs5_pad($text, $blocksize = 16) {
$pad = $blocksize - (strlen($text) % $blocksize);
return $text . str_repeat(chr($pad), $pad);
}
// Trims PKCS#5 padding
function pkcs5_unpad($text) {
$pad = ord($text{strlen($text)-1});
if ($pad > strlen($text)) {
return false;
}
if (strspn($text, chr($pad), strlen($text) - $pad) != $pad) {
return false;
}
return substr($text, 0, -1 * $pad);
}
// Private key
// Must be related to the public key, which was used in Delphi
// If you are going to use this example, you need to replace this contant
// This is just an example. Real program would store key in a file
$PrivateKey = <<<EOD
-----BEGIN PRIVATE KEY-----
MIIJRAIBADANBgkqhkiG9w0BAQEFAASCCS4wggkqAgEAAoICAQDkIEGZvxCUNW3u
...
X+VZcAA+CUNp1xYUuep0UoeqYtfzTuKvinNo3PWAItyqtCJNcdNWDzbWRizMEPJK
N9AYlDckmF/X/1cZxWNyiltZZ542kezn
-----END PRIVATE KEY-----
EOD;
// Read arguments (encrypted session key and encrypted data)
$EncodedKey = $_GET['key'];
$EncodedData = $_GET['data'];
// Convert text to bytes
$EncryptedKey = base64_decode($EncodedKey);
$EncryptedData = base64_decode($EncodedData);
// Convert little-endian (Windows/EurekaLog/WinCrypt) to big-endian (PHP/OpenSSL)
$EncryptedKey = strrev($EncryptedKey);
// Decrypt the session key
if (!openssl_private_decrypt($EncryptedKey, $Key, $PrivateKey)) {
echo('{ "error": ' . json_encode('openssl_private_decrypt: ' . openssl_error_string()) . ' }');
die();
}
// Decrypt the request
$Data = pkcs5_unpad(mcrypt_decrypt(MCRYPT_TWOFISH, $Key, $EncryptedData, MCRYPT_MODE_ECB));
// Convert JSON-text to JSON-object
$Data = json_decode($Data, true);
// Simple checks to validate the request (just an example)
$Request = $Data['request'];
if (($Request['version'] < 1) || ($Request['version'] > 1)) {
echo('{ "error": "Unsupported request" }');
die();
}
// Are we being asked for a license?
if ($Request['type'] == 'license') {
// Validate the requester
$User = $Data['user'];
// Just an example
$OK = (($User['login'] == 'input-from-edit1') && ($User['password'] == 'input-from-edit2'));
if ($OK) {
// Somehow obtain the license for the requester
// Real app would probably send encrypted license
$License = 'This is just an example license';
// Sign the license
openssl_sign($License, $Signature, $PrivateKey, OPENSSL_ALGO_SHA1);
// Convert big-endian (PHP/OpenSSL) to little-endian (Windows/EurekaLog/WinCrypt)
$Signature = strrev($Signature);
// Convert bytes to text
$EncodedLicense = base64_encode($License);
$EncodedSignature = base64_encode($Signature);
// Return license and digital signature to a caller
echo('{ "license": ' . json_encode($EncodedLicense) . ', "signature": ' . json_encode($EncodedSignature) . ' }');
} else {
echo('{ "error": "Access Denied" }');
die();
}
}
echo('{ "error": "Unsupported request" }');</pre>GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-50606176072327525692021-05-26T12:38:00.002+01:002021-05-26T12:38:20.501+01:00Introducing GitLab and GitHub support in EurekaLogEurekaLog 7 now supports integrated bug trackers of <a title="GitLab is the open
DevOps platform" href="https://about.gitlab.com/">GitLab</a> and <a title="GitHub - Where the world builds software" href="https://github.com/">GitHub</a>.<br />
<a name='more'></a><br />
GitLab is a web-based DevOps lifecycle tool that provides a Git-repository manager providing wiki, issue-tracking and continuous integration and deployment pipeline features, using an open-source license, developed by GitLab Inc.<br />
<br />
GitHub, Inc. is a provider of Internet hosting for software development and version control using Git. It offers the distributed version control and source code management (SCM) functionality of Git, plus its own features. It provides access control and several collaboration features such as bug tracking, feature requests, task management, continuous integration and wikis for every project.<br />
<br />
EurekaLog 7.9.4.0 introduces integration support with GitLab and GitHub. It's available in all supported IDEs. It's implemented very similary to other supported bug trackers. <a href="https://www.eurekalog.com/help/eurekalog/send_options_gitlab.php" title="GitLab options">You can configure sending to GitLab in EurekaLog project options</a>, <a href="https://www.eurekalog.com/help/eurekalog/send_options_github.php" title="GitHub options">as well as configure sending to GitHub</a>. We also have a <a title="GitLab Setup" href="https://www.eurekalog.com/help/eurekalog/gitlab_setup.php">GitLab configuration guide</a> and a <a title="GitHub Setup" href="https://www.eurekalog.com/help/eurekalog/github_setup.php">GitHub configuration guide</a>.<br />
<br />
Once you've set all things up - your application will submit bug reports to your GitLab or GitHub installation. New issues will be created for new bugs. Duplicate reports will be merged.<br />
<br />
Note that integrated bug trackers of GitLab and GitHub are quite feature-limited (when compared to standalone bug trackers). We recommend to consider <a title="GitLab: External issue tracker" href="https://docs.gitlab.com/ee/integration/external-issue-tracker.html">using external bug tracker</a> (such as <a href="https://www.eurekalog.com/help/eurekalog/jira.php" title="EurekaLog: Jira">Jira</a>, for example) for GitLab.<br />
<br />
P.S. Also note that EurekaLog supports BitBucket, because BitBucket uses Jira as its bug tracker, and <a href="https://www.eurekalog.com/help/eurekalog/send_options_jira.php" title="EurekaLog: Jira send options">EurekaLog has support for Jira</a>.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-21096295342597110582021-02-22T18:02:00.000+00:002021-04-07T11:49:40.497+01:00EurekaLog + VirusTotal = unforeseen consequencesWe were contacted by a person who reported <a href="https://www.half-life.com/en/halflife" title="Unforeseen consequences">unforeseen consequences</a> of uploading EurekaLog-enabled application to the <a href="https://www.virustotal.com/" title="www.virustotal.com">VirusTotal</a> service.<br />
<br />
It was like this: the client compiled an application with EurekaLog. The application was configured to send bug reports by e-mail. He uploaded the compiled application to the VirusTotal website, and got a scan result that everything is fine. <br />
<br />
So far, everything is quite typical. The strange things started the next day, when the client received an e-mail with bug report from EurekaLog. The weird thing was that the client did not launch the application, and he did not distribute/deploy it. And the report itself looked... unusual.<br />
<a name='more'></a><br />
In particular, the executable file was renamed to a random set of letters, as well as the username and computer name. There was nothing suspicious in the list of modules and processes, and in general the machine seemed "bare". The only aspect that stood out was the loaded <code>pancore.dll</code>, which has created one thread. Google suggests that <code>pancore.dll</code> is a part of Oracle AutoVue - an enterprise solution for visualizing and viewing CAD and similar data.<br />
<br />
The answer to the "riddle" came later. This is what the results of analyzing the file looked like during the first check:
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhk0irmZnrzDB6GSlgLHfcrFTtCPZR0ZSoJOUVCxsScP8Sjhf3xYmrvsJsHm5GlYBBjW_R_JE13w0JN-s1aTRXMgsuLuzAqExPe-JvVdg8DRjNh3-PJgfwOTUydB-ID-3s5trq1VeJBlvr5/s1786/u1.png" style="margin-left: 1em; margin-right: 1em;" title="Просмотр оригинала картинки"><img border="0" data-original-height="602" data-original-width="1786" height="216" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhk0irmZnrzDB6GSlgLHfcrFTtCPZR0ZSoJOUVCxsScP8Sjhf3xYmrvsJsHm5GlYBBjW_R_JE13w0JN-s1aTRXMgsuLuzAqExPe-JvVdg8DRjNh3-PJgfwOTUydB-ID-3s5trq1VeJBlvr5/w640-h216/u1.png" width="640" /></a></div>
<br />
And here is what the site shows when re-uploading the same file a day later (after receiving the "mysterious" report): <br /><br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhY0pV17Zf0UptI4ZJiRJZFSON2dNDufdCJL59MOe5R2DTPV8XCSaly1gJIRFr1aAXnM-EOjpN2Uvqv4z0g48ViX3wftp6rwYML6qvzw0hkd2NgCJzOF3KFJZoQJh7_Xgf_GJvrowz77eO/s1765/u2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="876" data-original-width="1765" height="318" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhY0pV17Zf0UptI4ZJiRJZFSON2dNDufdCJL59MOe5R2DTPV8XCSaly1gJIRFr1aAXnM-EOjpN2Uvqv4z0g48ViX3wftp6rwYML6qvzw0hkd2NgCJzOF3KFJZoQJh7_Xgf_GJvrowz77eO/w640-h318/u2.png" width="640" /></a></div>
<br />
As you can see, the scan results have changed: "interesting" behavior patterns have been added to the title, and new tabs have appeared in the full report, containing the analysis of the file's behavior: which files it opens, which URLs it visits, which registry keys it changes, which processes it launches, and so on.<br />
<br />
It turns out that VirusTotal runs uploaded apps in multiple virtual machines / sandboxes (aka multisandboxing) to determine details of its behavior. In particular, the file we uploaded has been verified in C2AE (presumably <a href="https://capesandbox.com/" title="capesandbox.com">CAPE Sandbox</a>), Sysinternals Sysmon tool, and VirusTotal's own sandbox: Jujubox.<br />
<br />
This feature has existed in VirusTotal since 2012, when <a href="https://blog.virustotal.com/2012/07/virustotal-behavioural-information.html" title="https://blog.virustotal.com/2012/07/virustotal-behavioural-information.html">they used VirusTotal Cuckoofork</a> - a clone of <a href="https://cuckoosandbox.org/" title="cuckoosandbox.org">CuckooBox</a>. <a href="https://blog.virustotal.com/2017/11/malware-analysis-sandbox-aggregation.html" title="Malware analysis sandbox aggregation">VirusTotal has launched multi-sandbox</a> in 2017, and <a href="https://blog.virustotal.com/2019/10/in-house-dynamic-analysis-virustotal-jujubox.html" title = "Revamping in-house dynamic analysis with VirusTotal Jujubox Sandbox">Cuckoofork was replaced with the new Jujubox Sandbox</a> in 2019.<br />
<br />
It is not hard to figure out that EurekaLog's report is a result of executing the uploaded file in one of these sandboxes (presumably - Jujubox). Now getting a "sudden" report does not seem so surprising anymore.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-50263224731385594452021-02-03T23:39:00.003+00:002023-05-30T08:58:05.099+01:00EurekaLog causes EOutOfResources (Out of system resources)?We were contacted by a person who complained that his application was working fine until he added EurekaLog to it. An "Out of system resources" exception was raised after adding EurekaLog to project. The exception occurred inside the <code>OutOfResources</code> helper function from the <code>Vcl.Graphics</code> unit.<br />
<a name='more'></a><br />
A call stack for the exception looked like this:
<ul>
<li>Vcl.Graphics.OutOfResources</li>
<li>Vcl.Graphics.GDIError</li>
<li>Vcl.Graphics.GDICheck</li>
<li>Vcl.Graphics.TransparentStretchBlt</li>
<li>Vcl.Graphics.TBitmap.Draw</li>
<li>Vcl.Graphics.TCanvas.Draw</li>
<li>SomeComponent.TSomeDBGrid.DrawCell</li>
<li>Vcl.Grids.DrawCells</li>
<li>Vcl.Grids.TCustomGrid.Paint</li>
<li>Vcl.Controls.TCustomControl.PaintWindow</li>
<li>Vcl.Controls.TWinControl.PaintHandler</li>
<li>Vcl.Controls.TWinControl.WMPrintClient</li>
<li>...</li>
</ul>
<br />
The exception itself is raised by this function:
<pre class="brush:delphi">procedure OutOfResources;
begin
raise EOutOfResources.Create(SOutOfResources);
end;</pre>
Which in turn is called from:
<pre class="brush:delphi">procedure GDIError;
const
BufSize = 256;
var
ErrorCode: Integer;
Buf: array [Byte] of Char;
begin
ErrorCode := GetLastError;
if (ErrorCode <> 0) and (FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM, nil, ErrorCode, LOCALE_USER_DEFAULT, Buf, BufSize, nil) <> 0) then
raise EOutOfResources.Create(Buf)
else
OutOfResources;
end;
function GDICheck(Value: THandle): THandle;
begin
if Value = 0 then
GDIError;
Result := Value;
end;</pre>
<br />
<blockquote>Note that this VCL implementation has a problem: regardless of the error, an exception of the <code>EOutOfResources</code> class is thrown - even if the error is not <code>ERROR_NOT_ENOUGH_MEMORY</code>, <code>ERROR_NO_SYSTEM_RESOURCES</code> (or a similar one). It would make more sense to raise something like <code>EInvalidGraphicOperation</code> for a general case, and raise <code>EOutOfResources</code> specifically only for errors of this type.</blockquote>
<br />
From the full EurekaLog bug report, it was clear that the memory and handles count are within a reasonable range, i.e. the problem is not a lack of memory. Which means that (most likely) <code>GetLastError</code> returned 0. Indeed, the line in <code>TransparentStretchBlt</code> that fails the <code>GDICheck</code> check looks like this:
<pre class="brush:delphi">MemBmp := GDICheck(CreateCompatibleBitmap(SrcDC, SrcW, SrcH));</pre>
You can see from <a href="https://docs.microsoft.com/en-us/windows/win32/api/wingdi/nf-wingdi-createcompatiblebitmap" title="MSDN: CreateCompatibleBitmap Function">the documentation</a> that the <code>CreateCompatibleBitmap</code> function does not set the <code>GetLastError</code> value on failure.<br />
<br />
However, there are not many reasons for the function to fail: either wrong arguments were passed to it, or it ran out of memory to create a bitmap. Note that running out of memory is also possible if <code>SrcW</code> and/or <code>SrcH</code> are trashed in such way, so these values become "too large". So while we don't know the exact reason for <code>CreateCompatibleBitmap</code>'s failure, we can assume that the problem is in its arguments.<br />
<br />
The <code>SrcDC</code>, <code>SrcW</code>, and <code>SrcH</code> values are arguments of the <code>TransparentStretchBlt</code> function and come from <code>TBitmap.Draw</code>:
<pre class="brush:delphi">TransparentStretchBlt
(ACanvas.FHandle, Left, Top, Right - Left, Bottom - Top,
Canvas.FHandle { SrcDC }, 0, 0,
FDIB.dsbm.bmWidth { SrcW }, FDIB.dsbm.bmHeight { SrcH },
MaskDC, 0, 0);</pre>
Where <code>Canvas</code> is the <code>FCanvas</code> field of <code>TBitmap</code> created on demand, and <code>FDIB</code> is the <code>FImage: TBitmapImage</code> <code>TBitmap</code>'s field. Thus, all arguments (<code>SrcDC</code>, <code>SrcW</code>, and <code>SrcH</code>) come to the <code>TransparentStretchBlt</code> function from fields of the <code>TBitmap</code> class object.<br />
<br />
Therefore the <code>TBitmap</code> that <code>TSomeDBGrid.DrawCell</code> is trying to draw is corrupted. Since the exception does not occur without EurekaLog, but does happen with EurekaLog, the <code>TBitmap</code>'s memory contents is changed when EurekaLog is enabled. The most likely explanation for this behavior is a "use after free" bug. Without debugging tools in the program: a code can access an already deleted <code>TBitmap</code> and "successfully" perform an operation with it - since the memory of freed objects is not physically deleted, but only marked as "free", without changing its content. However, when addeding EurekaLog to an application, its default configuration includes memory checks that erase the memory when it is freed.<br />
<br />
You can check this hypothesis by <a href="https://www.eurekalog.com/help/eurekalog/memory_leaks_page.php" title="EurekaLog Help: Memory Options">changing the "When memory is released" option to "Do nothing" and unchecking the "Catch memory problems" option</a>. If the <code>EOutOfResources</code> exception disappears after changing these options, then the code contains a "use after free" bug. Most likely the bug is in <code>SomeComponent</code>'s code, but there is a small nonzero chance that the client has found a bug in the VCL itself - similar to <a href="https://blog.eurekalog.com/2022/07/eurekalog-erases-my-bitmap.html" title="EurekaLog erases my bitmap? (even VCL has bugs)">this one</a>.<br />
<br />
Unfortunately, we have not received a response from the client.<br />
<br />
<a href="https://blog.eurekalog.com/search/label/Stories" title="EurekaLog Blog: Stories">Read more stories like this one</a> or <a href="https://www.eurekalog.com/casestudies.php" title="Case Studies and User Reviews">read feedback from our customers</a>.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-66723305453845137162021-02-01T13:58:00.002+00:002021-02-01T13:59:27.893+00:00EurekaLog 2020 Survey ResultsRecently we ran <a href="https://blog.eurekalog.com/2020/12/eurekalog-survey-2020.html" title="Win EurekaLog license by taking a simple survey">a survey of EurekaLog users</a>. We asked questions that will help us improve EurekaLog.<br />
<br />
Survey participants were automatically added to our big giveaway drawing.<br />
<br />
Today, we picked 3 lucky winners at random to receive some valuable prizes. <a name='more'></a>Here they are:
<ul>
<li><b>Gernot Baecker</b> (finished survery on 2020-12-28, UTC)</li>
<li><b>Jim Carter</b> (finished survery on 2021-01-03, UTC)</li>
<li><b>Attila Kovacs</b> (finished survery on 2020-12-25, UTC)</li>
</ul>
We would like to send out a big Thank You to everyone that sent in a survey! Your answers are valuable to us in making EurekaLog a better product.<br />
<br />
P.S. Winners will soon be contacted by e-mail to select their prizes. If you believe you are a winner, and you will not receive a e-mail from us within reasonable amount of time - check your spam folder, be sure <code>@eurekalog.com</code> is whitelisted. Or you can <a href="https://www.eurekalog.com/support.php" title="Contact Support">contact us</a> to confirm.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-84792577051430374892020-12-25T01:22:00.000+00:002020-12-25T01:22:52.899+00:00Win EurekaLog license by taking a simple surveyYou can win a free EurekaLog license or extension to maintenance period of your existing license.<br />
<a name='more'></a><br />
Rules are simple: you help us to improve EurekaLog by taking part in a survey. Filling out the survey will take just a few minutes of your time. 3 random participants will be selected after the survey ends. <br />
<ul>
<li>Winners will receive one Single Developer Enterprise license if they do not already have a license.</li>
<li>Or, a winner will receive a free 2 year extension to maintenance period of his existing license.</li>
</ul>
<br />
You can <a href="https://www.eurekalog.com/getfile.php?id=83" title="EurekaLog Survey 2020">take the survey here</a>.<br />
<br />
Survey will end on January 31, 2021. Winners will be announced on February 1, 2021. You can take the survey only once. Duplicates will be deleted. Winners will be contacted by e-mail.
GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-73100437218855592072020-12-24T02:01:00.002+00:002023-09-04T23:08:48.711+01:00Even empty applications have bugsWe were contacted by a person who claimed to have found a bug in EurekaLog. He justified this statement as follows: if you create a new DataSnap application and add EurekaLog to it, the application will crash with Access Violation on exit.<br />
<a name='more'></a><br />
The investigation revealed the following:
<ul>
<li>The <code>ServerContainerUnit1</code> data module contains a <code>DSServer1</code> server component;</li>
<li>The same data module contains two auxiliary components: <code>DSTCPServerTransport1</code> and <code>DSServerClass1</code>;</li>
<li>Both <code>DSTCPServerTransport1</code> and <code>DSServerClass1</code> specify the <code>DSServer1</code> as a "Server";</li>
<li>When the application exits, the <code>ServerContainerUnit1</code> data module will be destroyed;</li>
<li>Destroying a data module means destroying all components inside - including <code>DSTCPServerTransport1</code> and <code>DSServerClass1</code>.<br />
<br />
Here's where it happens:
<code>
<ul>
<li>TDSServerClass.Destroy</li>
<li>TComponent.DestroyComponents</li>
<li>TDataModule.Destroy</li>
</ul>
<br />
<ul>
<li>TDSTCPServerTransport.Destroy</li>
<li>TComponent.DestroyComponents</li>
<li>TDataModule.Destroy</li>
</ul> </code>
However, neither <code>TDSServerClass.Destroy</code> nor <code>TDSTCPServerTransport.Destroy</code> notify the server that they are being destroyed. As a result, <code>DSServer1</code> continues to store reference to the (already deleted) <code>DSTCPServerTransport1</code> and <code>DSServerClass1</code>;</li>
<li>The <code>ServerContainerUnit1</code> data module continues to clean up;</li>
<li>It is now time to remove <code>DSServer1</code>. Specifically, the <code>DSServer1</code> tries to stop all registered transports:
<code>
<ul>
<li>TDSCustomServer.StopTransports</li>
<li>TDSCustomServer.Stop</li>
<li>TDSServer.Stop</li>
<li>TDSServer.Destroy</li>
<li>TObject.Free</li>
<li>TComponent.DestroyComponents</li>
<li>TDataModule.Destroy</li>
</ul>
</code>
which will cause Access Violation, since the objects for these transports have already been deleted.</li>
</ul>
<br />
As you can see, it is a bug in DataSnap, not in EurekaLog. EurekaLog only revealed this bug. Indeed, when EurekaLog is not enabled, <code>DSServer1</code> can "successfully" call the <code>StopTransports</code> method, since memory of already deleted objects will not be changed, therefore calls of methods of already deleted objects inside <code>StopTransports</code> will be "successful".<br />
<br />
Note that this error has been sitting in DataSnap for ages and no one fixes it - precisely because <a href="https://blog.eurekalog.com/2023/04/librariescomponentsarenottested.html" title="Many libraries/components are not tested for memory bugs">there are no means to detect it in a naked application (without additional debugging tools)</a>. There are similar problems in the empty Fire Monkey (FMX) application, as well as in many standard Delphi components.<br />
<br />
Specifically, this error can be worked around if you force <code>DSServer1</code> to be deleted first:
<pre class="brush:delphi">procedure TServerContainer1.DataModuleDestroy (Sender: TObject);
begin
FreeAndNil (DSServer1);
end;</pre>
Other errors of a similar nature should be investigated separately.<br />
<br />
If you cannot fix the error, you can always turn off memory checks in EurekaLog, although this is not recommended.<br />
<br />
<a href="https://blog.eurekalog.com/search/label/Stories" title="EurekaLog Blog: Stories">Read more stories like this one</a> or <a href="https://www.eurekalog.com/casestudies.php" title="Case Studies and User Reviews">read feedback from our customers</a>.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.comtag:blogger.com,1999:blog-6789056281440007248.post-20222589192993533692020-11-19T07:31:00.005+00:002021-04-07T11:50:03.031+01:00EurekaLog causes Integer Overflow?We were contacted by a person who complained that his application was working fine until he added EurekaLog to it. An Integer Overflow exception was raised after adding EurekaLog to project. The exception occurred inside the <code>_UStrCatN</code> function (it is a function for concatenating multiple strings in RTL).<br />
<a name='more'></a><br />
The <code>_UStrCatN</code> function is written in assembler, but if you delve into the meaning of the checks, you get something like this:
<pre class="brush:delphi"> DestLen := {... the length of the resulting string is calculated ...};
if DestLen < 0 then
_IntOver;</pre>
where <code>_IntOver</code> is the RTL's function that raises the Integer Overflow exception.<br />
<br />
What's happening? How can a string length be negative? Is this a bug in EurekaLog?<br />
<br />
The specified check inside <code>_UStrCatN</code> is designed to limit strings to 2 GB of memory: if concatenation's result is more than 2 GB in length, then a literal integer overflow will occur, so the final length will become negative. Thus, Integer Overflow exception may occur when combining large strings (when result is too large).<br />
<br />
But what does EurekaLog have to do with it then? And how can the check trigger when we contatenate short strings? (the client confirmed this with a log)<br />
<br />
Such a "false-positive" response is possible if you are performing an operation on already deleted string.<br />
<br />
Look at this code:<br />
<pre class="brush:delphi">var
Marker: String;
function ReadLine: String;
begin
// ...
Marker := { ... };
// ...
end;
begin
// ...
Data := Data + Marker + ReadLine;
// ...
end;</pre>
Do you see a problem in this code?<br />
<br />
To understand the problem, you need to know how the "<code>Data: = Data + Marker + ReadLine;</code>" line is executed. It looks something like this in pseudo-code:<br />
<pre class="brush:delphi">Param0 := Pointer(Data);
Param1 := Pointer(Marker);
Param2 := Pointer(ReadLine);
_UStrCatN(Data, [Param0, Param1, Param2]);</pre>
In other words, the operator stores pointers to arguments sequentially before calling the function.<br />
<br />
So, here's the bug: the statement stores a pointer to the <code>Marker</code> string, but the <code>Marker</code> string is changed inside the <code>ReadLine</code> function. This means that the stored pointer will point to the old string. Thus, the already deleted string will be sent to the <code>_UStrCatN</code> function.<br />
<br />
Note that this bug is not a "problem" without EurekaLog in the project. Indeed, a deleted memory is simply marked as "free", but its content is not cleared. This means that <code>_UStrCatN</code> will successfully concatenate with the already deleted string. And the result of the operation will most likely be correct. E.g. there is a bug in the code, but it is completely invisible, since the program functions perfectly.<br />
<br />
The situation changes radically if EurekaLog (or any other tool for debugging memory problems) is added to the project. By default, memory checks are enabled in EurekaLog. This means that a deleted memory will be cleaned up. Typically, this is done with a template like <code>DEADBEEF</code>. Note that the Integer representation of <code>DEADBEEF</code> is negative (equal to -559038737). Therefore adding lengths of several short strings to this number will also produce a negative number.<br />
<br />
In other words, if EurekaLog is added to the project, then the operation with the already deleted string will no longer be successful. A previously hidden bug is now visible.<br />
<br />
<a href="https://blog.eurekalog.com/search/label/Stories" title="EurekaLog Blog: Stories">Read more stories like this one</a> or <a href="https://www.eurekalog.com/casestudies.php" title="Case Studies and User Reviews">read feedback from our customers</a>.GunSmokerhttp://www.blogger.com/profile/15611696588191431330noreply@blogger.com