How to get a list of used units with DelphiAST
Getting a list of used units from the source code can be useful in many code automation tasks. There are a number of tools that can help you to investigate what dependencies you have in your project. In this post I will show how you can write your own simple dependency analysis tool using DelphiAST library.
This demo program will count how many times each unit is used by other units and will consist of two steps:
- Get a list of units from a DPR file.
- Go through this list, analyze uses statements and count used units.
To keep demo as simple as possible, it will ignore a unit if its path was not mentioned in a DPR.
As Delphi strings and chars are Unicode, before starting to parse any file we have to convert it to Unicode. This function reads a file to a string, converting it to Unicode if needed.
function TForm1.ReadFileToString(const FileName: string): string;
var
Buffer: TBytes;
Content: TMemoryStream;
Encoding: TEncoding;
PreambleSize: Integer;
begin
Encoding := nil;
Content := TMemoryStream.Create;
try
Content.LoadFromFile(FileName);
SetLength(Buffer, Content.Size);
Content.Read(Buffer, 0, Content.Size);
PreambleSize := TEncoding.GetBufferEncoding(Buffer, Encoding, TEncoding.Default);
Result := Encoding.GetString(Buffer, PreambleSize, Content.Size - PreambleSize);
finally
Content.Free;
end;
end;
And now we’re getting to the fun part, to parsing.
function TForm1.Parse(const Content: string): string;
var
ASTBuilder: TPasSyntaxTreeBuilder;
StringStream: TStringStream;
SyntaxTree: TSyntaxNode;
begin
Result := '';
StringStream := TStringStream.Create(Content, TEncoding.Unicode);
try
StringStream.Position := 0;
ASTBuilder := TPasSyntaxTreeBuilder.Create;
try
ASTBuilder.InitDefinesDefinedByCompiler;
SyntaxTree := ASTBuilder.Run(StringStream);
try
Result := TSyntaxTreeWriter.ToXML(SyntaxTree);
finally
SyntaxTree.Free;
end;
finally
ASTBuilder.Free;
end;
finally
StringStream.Free;
end;
end;
This function returns parser’s output as an XML in a string. The coolest fact about XML is that it makes possible to query the syntax tree using XPath. You will see some examples below.
Let’s see how a DPR syntax tree looks like.
<UNIT line="1" col="1" name="UnitCounter">
<USES line="3" col="1">
<UNIT line="4" col="3" name="Vcl.Forms"/>
<UNIT line="5" col="3" name="frmMain" path="frmMain.pas">
<EXPRESSION line="5" col="14">
<LITERAL line="5" col="35" type="string" value="frmMain.pas"/>
</EXPRESSION>
</UNIT>
</USES>
<STATEMENTS end_line="14" begin_line="10" end_col="1" begin_col="3">
<CALL line="10" col="3">
<DOT>
<IDENTIFIER line="10" col="3" name="Application"/>
<IDENTIFIER line="10" col="15" name="Initialize"/>
</DOT>
</CALL>
<ASSIGN line="11" col="3">
<LHS>
<DOT>
<IDENTIFIER line="11" col="3" name="Application"/>
<IDENTIFIER line="11" col="15" name="MainFormOnTaskbar"/>
</DOT>
</LHS>
<RHS>
<EXPRESSION line="11" col="36">
<IDENTIFIER line="11" col="36" name="True"/>
</EXPRESSION>
</RHS>
</ASSIGN>
<CALL line="12" col="3">
<CALL>
<DOT>
<IDENTIFIER line="12" col="3" name="Application"/>
<IDENTIFIER line="12" col="15" name="CreateForm"/>
</DOT>
<EXPRESSIONS>
<EXPRESSION line="12" col="26">
<IDENTIFIER line="12" col="26" name="TForm1"/>
</EXPRESSION>
<EXPRESSION line="12" col="34">
<IDENTIFIER line="12" col="34" name="Form1"/>
</EXPRESSION>
</EXPRESSIONS>
</CALL>
</CALL>
<CALL line="13" col="3">
<DOT>
<IDENTIFIER line="13" col="3" name="Application"/>
<IDENTIFIER line="13" col="15" name="Run"/>
</DOT>
</CALL>
</STATEMENTS>
</UNIT>
Definetely, we are interested in UNIT/USES subtree.
Thats how it looks like in Delphi, it couldn’t be easier.
procedure TForm1.GetFileList(const RootFolder, ParsedDpr: string;
const Files: TStringList);
var
XmlDoc: IXMLDOMDocument2;
UnitNodes: IXMLDOMNodeList;
PathAttrNode: IXMLDOMNode;
I: Integer;
FileName: string;
begin
Files.Clear;
XmlDoc := CoDOMDocument60.Create;
XmlDoc.SetProperty('SelectionLanguage', 'XPath');
XmlDoc.validateOnParse := False;
XmlDoc.preserveWhiteSpace := False;
XmlDoc.resolveExternals := False;
XmlDoc.loadXML(ParsedDpr);
// select all units nodes
UnitNodes := XmlDoc.documentElement.selectNodes('/UNIT/USES/UNIT');
for I := 0 to UnitNodes.length - 1 do
begin
// if 'path' attribute exists, then use it
PathAttrNode := UnitNodes.item[I].attributes.getNamedItem('path');
if Assigned(PathAttrNode) then
FileName := PathAttrNode.nodeValue
else
FileName := UnitNodes.item[I].attributes.getNamedItem('name').nodeValue + '.pas';
FileName := ExpandFileName(TPath.Combine(RootFolder, FileName));
// ignore units that placed somewhere on the search path
if FileExists(FileName) then
Files.Add(FileName);
end;
end;
Now, when we have a list of units file names we can start to calculate.
That’s how a unit syntax tree looks like. I have skipped not interesting but too long parts. You can use DelphiAST demo application to see a full syntax tree for any unit.
<UNIT line="1" col="1" name="frmMain">
<INTERFACE line="3" col="1">
<USES line="5" col="1">
<UNIT line="6" col="3" name="Winapi.Windows"/>
<UNIT line="6" col="19" name="Winapi.Messages"/>
<UNIT line="6" col="36" name="System.SysUtils"/>
<UNIT line="6" col="53" name="System.Variants"/>
<UNIT line="6" col="70" name="System.Classes"/>
<UNIT line="6" col="86" name="Vcl.Graphics"/>
<UNIT line="7" col="3" name="Vcl.Controls"/>
<UNIT line="7" col="17" name="Vcl.Forms"/>
<UNIT line="7" col="28" name="Vcl.Dialogs"/>
<UNIT line="7" col="41" name="Vcl.StdCtrls"/>
<UNIT line="7" col="55" name="Generics.Collections"/>
</USES>
<SKIPPED>..</SKIPPED>
</INTERFACE>
<IMPLEMENTATION line="27" col="1">
<USES line="29" col="1">
<UNIT line="30" col="3" name="DelphiAST"/>
<UNIT line="30" col="14" name="DelphiAST.Classes"/>
<UNIT line="30" col="33" name="DelphiAST.Writer"/>
<UNIT line="30" col="51" name="MSXML2_TLB"/>
<UNIT line="30" col="63" name="IOUtils"/>
</USES>
<SKIPPED>..</SKIPPED>
</IMPLEMENTATION>
</UNIT>
We are interested in particular name attribute of /UNIT/INTERFACE/USES/UNIT and /UNIT/IMPLEMENTATION/USES/UNIT nodes.
procedure TForm1.CalculateUnits(const ParsedUnit: string;
const Stats: TDictionary<string, Integer>);
var
XmlDoc: IXMLDOMDocument2;
UnitNodes: IXMLDOMNodeList;
I: Integer;
UnitName: string;
begin
XmlDoc := CoDOMDocument60.Create;
XmlDoc.SetProperty('SelectionLanguage', 'XPath');
XmlDoc.validateOnParse := False;
XmlDoc.preserveWhiteSpace := False;
XmlDoc.resolveExternals := False;
XmlDoc.loadXML(ParsedUnit);
UnitNodes := XmlDoc.documentElement.selectNodes('/UNIT/INTERFACE/USES/UNIT/@name|/UNIT/IMPLEMENTATION/USES/UNIT/@name');
for I := 0 to UnitNodes.length - 1 do
begin
UnitName := UnitNodes.item[I].nodeValue;
if not Stats.ContainsKey(UnitName) then
Stats.Add(UnitName, 0);
Stats[UnitName] := Stats[UnitName] + 1;
end;
end;
And last, but not least, let’s put all parts together.
var
Units: TStringList;
FileName: string;
Stats: TDictionary<string, Integer>;
UnitCounter: TPair<string, Integer>;
begin
if OpenDialog.Execute then
begin
Units := TStringList.Create;
try
GetFileList(ExtractFilePath(OpenDialog.FileName),
Parse(ReadFileToString(OpenDialog.FileName)), Units);
Stats := TDictionary<string, Integer>.Create;
try
for FileName in Units do
CalculateUnits(Parse(ReadFileToString(FileName)), Stats);
memUnits.Clear;
for UnitCounter in Stats do
memUnits.Lines.Add(UnitCounter.Key + ' : ' + IntToStr(UnitCounter.Value));
finally
Stats.Free;
end;
finally
Units.Free;
end;
end;
That’s it. You can see an example of its output on the screenshot in the beginning of this post.
You can download the full source code here, DelphiAST is available on GitHub. This application is much less complicated than this kind of application is usually expected to be. DelphiAST is a powerful tool and it is a must have if you are going to automatically process your Delphi source code.