-
Notifications
You must be signed in to change notification settings - Fork 196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Building issue Win10x64 Python3.7.1 / x64 / VS2017 / QL1.15 / Boost 1.69 #252
Comments
Going through older posts, I have now installed cygwin with binutils pkg
However I still do have errors running the properly amended setup.py
Also I am not to sure about the following: thanks |
|
Windows instructions are pretty outdated. If you manage to build quantlib, we should update the docs. Regarding your last error, it's a bug in the setup file. You can comment out that line for now. |
I have the same issue, but according to your second post you seem to have solved this. How did you solve it? |
You need |
Yeah I'm finding the same issue here much later on MSVC 14.1 and using SDK 8.1 or 10.x. It really comes down to just a few issues apparently after I dug through the error messages - there's really not all that many. I can build a DLL but I doubt it works. Here's what has to be fixed (in Boost), and I suck at C++, so someone with more expertise would be ideal to work through these (what appear to be only 3) issues with the DLL not compiling properly on Windows: DLL Interfaces missing in \quantlib-1.22\ql\settings.hpp: Line Number
Anybody got any ideas? Someone who knows C++ well can probably fix the problematic issues above without much effort, which would get PyQL working on Windows machines. It will take me a long time, and I really want to use the PyQL interface. Too bad there isn't a .lib version of PyQL, that would solve everything. For you C++ experts you can just head over to my GitHub for updated instructions that actually work (as of July 2021) and a compiled QuantLib 1.22 using Boost 1.76.0 with MSVC 14.1 and both v8.1 and v10.x SDKs posted there under releases (QuantLib-x64-mt.lib) |
From what I remember, the issue is not so much building pyql on windows, which is easy enough, but having a working copy. If you try to run the tests, I suspect it fails pretty hard. The singleton used by the Settings class gets compiled differently on windows, and doesn't behave as singleton. |
Thanks thrasibule - are there any workarounds you've come across to correct the behavior introduced by the singleton not compiling as it should on Windows, and behaving like its not a singleton? Is there another compiler better suited to produce a working copy on Windows that avoids this issue? The project I'm working on must run on Windows, which wherein the problem lies, otherwise I'd have already done it all on my WSL2 Debian instance. |
Would this defeat the singleton issue on Windows? Without destroying the functionality of PyQL? If implemented into the code obviously (this example is supposed to follow thread-safe C++ 14 format):
I am not good at C++ so I wouldn't know where to put it (somewhere in settings.hpp I presume). However, it would be great if something simple like the above fixed PyQL issues on Windows. Then you could reach a much wider audience. There's also some older posts that may work, but I'm no expert: https://stackoverflow.com/questions/6915/thread-safe-lazy-construction-of-a-singleton-in-c?rq=1 Any feedback or assistance is greatly appreciated. |
Just to note - if these instructions are NOT followed: In the ql/settings.hpp file, update the Settings class defintion as following (line 37):
class __declspec(dllexport) Settings : public Singleton<Settings> { QuantLib compiles just fine on MSVC 14.1 as a DLL. Hmm... Is the above change really needed? |
Seems to me, with limited knowledge, the BOOST libraries definitions for singleton need to be changed and Boost recompiled for it to work on Windows x64 MSVC 14.1+. Using the C++ 11 standard the Meyer's singleton should work, although I found a website with numerous examples of how the boost code could be edited: This one in particular actually goes through the Boost code (and has thread safe implementations that work with any MSVC / C++ standard and multithreading): This also has some good examples: I'm not good with C++, but the templates defined are in the Boost source in numerous locations, and they seem to use quite a few implementations. I'm unsure which exact one is causing the issue with this working on Windows. |
@mrslezak I'm not sure the singleton code needs fixing per se. As far as I know, the Swig bindings to quantlib work fine on windows, so I think the C++ code is fine as it is. My guess is that the issue is more a bad interaction between cython and the MSVC compiler, that may be fixed by changing the compiler flags. |
@mrslezak I've spent a lot of time trying to understand what the exact issue was between the cython binding and the behaviour of the Settings singleton instantiated from the dll. It's been working with the above-mentioned patch for some time (IIRC prior to the migration to Python 3). If you're investing time looking into the issue, I would suggest starting from a clean slate and not look in the suggested workaround as it's pretty old, it's not working anymore. @thrasibule has a good suggestion: looking into the compiler flags is likely the right approach. |
Good point - the default flags are full optimization Ox + OpenMP maybe
that's too aggressive and mangling symbols and code too much. I haven't
looked much at the PyQL code - all the options models I wrote in Cython
prior I'd use the Windows OpenMP interface in the Cython code itself with
NumPy memory views, there has never been a faster C++ implementation I've
witnessed...
E.g.
cimport cython
cimport numpy as np
from cython cimport view
from cython.parallel cimport prange
from openmp cimport omp_get_max_threads
cpdef black76(double[::1] PnL, double[::1] F, double[::1] T, double[::1]
S, double[::1] vol, double[::1] r, int[::1] callput):
cdef int i, N
N=PnL.size
cdef double d1, d2
cdef int maxthreads = omp_get_max_threads()
for i in prange(N, nogil=True, num_threads=maxthreads,
schedule='static'):
d1=
d2=
PnL[I] =
return np.asarray(PnL)
In the .pyx file. With a cdef inline implementation of norm_cdf, this is
the fastest way to price an analytical option I've ever seen. If your
wrappers + Boost still maintain the NumPy memoryviews I would imagine the
speed could be similar...
Nonetheless I'm on a project that requires already established libraries
that have been tied out (not just by me)... So they would rather use a
slower implementation than validate all my code. I could never get STL
vectors running as fast as straight memoryviews; there's always a copy
operation it seems that can't be avoided in memory. Maybe PyQL has made a
Boost wrapper that avoids that.
Now I'll likely just give up I installed WSL2 Debian Buster, just need to
defeat the firewall placed on it so I can reach XFCE4. Or I'm stuck on the
command line, could be worse... If I do make any progress on the Windows
side I'll definitely contribute the fixes.
…On Mon, Jul 19, 2021, 3:09 AM Didrik Pinte ***@***.***> wrote:
@mrslezak <https://github.com/mrslezak> I've spent a lot of time trying
to understand what the exact issue was between the cython binding and the
behaviour of the Settings singleton instantiated from the dll. It's been
working with the above-mentioned patch for some time (IIRC prior to the
migration to Python 3). If you're investing time looking into the issue, I
would suggest starting from a clean slate and not look in the suggested
workaround as it's pretty old, it's not working anymore. @thrasibule
<https://github.com/thrasibule> has a good suggestion: looking into the
compiler flags is likely the right approach.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#252 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AECPVRWDTJWPKZQ6ZSKJY23TYPMT3ANCNFSM4GYT5ZVQ>
.
|
I think the equivalent to nm on Windows is dumpbin.exe ran from a Visual Studio Command Prompt. So in theory the nm could possibly be replaced with a different script that uses dumpbin, provided the output file matches what is inside the generated one on Linux to get all the right outputs from generate_symbols.py. I am curious why Quantlib is compiled with static Boost rather than Boost.dll. Only because of the MSVC compiler warnings when building QuantLib as a DLL. I mean it does look like some things would have to be modified in the source to do it that way, so maybe it isn't practical. Anyway, if anyone can provide a Linux example def file I'd appreciate it. Otherwise I'll be building PyQL on Linux and figure it out on my own. Eventually... Too busy at work to try now but it is on my todo list. Although I just found a merged commit to enable a safe singleton thread-safe class, this should be out in the next QuantLib release: lballabio/QuantLib@185c914 that was another issue faced on Windows MSVC builds. |
I just found a merged commit to enable a safe singleton thread-safe class, this should be out in the next QuantLib release: lballabio/QuantLib@185c914 |
Yes I posted that in another thread, or issue here; and tried building the dev build with the singleton changes 1.43-dev I believe, but see there is still an issue. I believe an alternative may be to generate symbols for a DLL on Windows (since it produces none by default on a DLL) is only solved by making a new Visual Studio project created using Cmake as mentioned here: https://cmake.org/cmake/help/latest/prop_tgt/WINDOWS_EXPORT_ALL_SYMBOLS.html although: for global data symbols, __declspec(dllimport) must still be used when compiling against the code in the .dll. I am not a good C++ programmer but the old Windows instructions do not appear to target a Global variable, although I could (and probably am) be mistaken. All other function symbols will be automatically exported and imported by callers. This simplifies porting projects to Windows by reducing the need for explicit dllexport markup, even in C++ classes. https://devblogs.microsoft.com/cppblog/cmake-support-in-visual-studio/ 2017 Visual Studio can use Cmake directly, and Cmake can generate .sln files. Hopefully this is helpful for someone who actually knows what they are doing. |
@mrslezak thanks for sharing your findings! On my end, I haven't touched the windows part of pyql for years and unfortunately have no time to go back to it right now. |
Hi,
Win10x64
Python 3.7.1
VS 2017
QL 1.15
Boost 1.69 (prebuilt boost_1_69_0-msvc-14.1-64.exe)
I followed the following instructions
https://pyql.readthedocs.io/en/latest/getting_started.html#installation-from-source-on-windows
1/ Step f.
I get lots of warnings although VS says the .dll has been built with success.
Why so many warnings ? Can we fix this ?
2/ Step g.
I struggle generating the .def file
As I get MSVC14.1 I had to do the following amendment %d1 instead of %d0 to get the correct folder
input_directory = r"C:\lib\QuantLib-1.15\build\vc%d1\%s\Release" % ((VC_VERSION, ARCH))
C:\lib\QuantLib-1.15\build\vc141\x64\Release
However, at this stage I get the following errors
Any idea ?
The text was updated successfully, but these errors were encountered: