Call dll function in NSIS and return boolean

Question

I tried to call the function in my dll, but it failed. Here is my NSIS script:

!include "LogicLib.nsh"
Section "MyTest"
Strcpy $0 "abc"
System::Call 'MyDll::FindSomething(t, b) i(r0, false) .r1'

${If} $1 == true
    MessageBox MB_OK "1"
${ElseIf} $1 == false
    MessageBox MB_OK "0"
${EndIf}
SectionEnd

The FindSomething function should return boolean value true or false. The result of my script show nothing after executing.

Please include the C definition when asking System::Call questions. — Anders

Anders Anders · Accepted Answer · 2019-02-26T20:57:58

You are not extracting MyDll.dll?
false is not a supported keyword, use 0.

Without the C/C++ declaration it is hard to give a full example but I can try:

Section
InitPluginsDir
File "/oname=$PluginsDir\MyDll.dll" "c:\myfiles\MyDll.dll" ; Extract

System::Call 'KERNEL32::AddDllDirectory(w "$PluginsDir")' ; Make sure we are allowed to load from here

System::Call 'KERNEL32::LoadLibrary(t "$PluginsDir\MyDll.dll")p.r9'
MessageBox mb_OK "Loaded MyDll at address $9" ; This should not be 0!

; bool __cdecl FindSomething1(char* p1, bool p2):
StrCpy $0 "abc"
System::Call 'MyDll::FindSomething1(m r0, b 0)b.r1 ?c'
MessageBox mb_OK "Returned $1"

; int WINAPI FindSomething2(LPTSTR p1, BOOL p2):
StrCpy $0 "abc"
System::Call 'MyDll::FindSomething2(t r0, i 0)i.r1'
MessageBox mb_OK "Returned $1"
; ${If} $1 <> 0 ...

System::Call 'KERNEL32::FreeLibrary(p r9)'
SectionEnd

Only very recent versions of the System plug-in supports the b type but you can just use i in most places anyway.

Call dll function in NSIS and return boolean

1 Answers