[archived] Previous, now Deprecated Whonix Windows Installer Testing

The new tool is called pzlib, it is open source, it decompresses zlib streams in files and makes them bigger so we can compress them later with better compressors. In the end, it restores the original file. It has 3 versions, the newer ones were buggy but this one turned out to be stable and creates reproducible results. Similar tools were usually very slow. When restoring files, pzlib supports multi-threading to speed up the operation. By default it uses 50% of the available cores.

Source code and binaries of pzlib are available here:

http://fileforums.com/showthread.php?t=97888

You need to be logged in to download, you can use these:

username: bugmenot_
password: bugmenot

Let’s create the same whonix.tar file

touch -d “2000-01-01 00:00:00 UTC” gateway.ova
touch -d “2000-01-01 00:00:00 UTC” workstation.ova
7za a -ttar whonix.tar gateway.ova workstation.ova

pzlib should create a 8,69 GB whonix.pz file out of 3,89 GB

pzlib e -v whonix.tar -o whonix.pz

srep will turn this into a 2,62 GB whonix.srep file

srep -m4f -hash- whonix.pz whonix.srep

let’s compress this file with xz using 7za to get to 1,20 GB

7za a -txz whonix.xz whonix.srep

Note: It is possible to lower the final size about 50 MB more but that requires a 64 bit 7za executable when compressing due to memory requirements, which means some people wouldn’t be able to reproduce the archive (with 32 bit OS/VM or low RAM) but it doesn’t change decompression requirements much.

These are my SHA256 hashes:

whonix.tar - 3,89 GB
F2265763F18717328E10FB8FA7FBC589B6E4D8C84F3DFEBDFE69213C51108557
whonix.pz - 8,69 GB
2977DF5AF4FF54DE27EAA54B4E82BB3246F9D4C2DD0EF9C63C3C6D63B2182C29
whonix.srep - 2,62 GB
F5B787CD063F876DB2FAE7D6764A0273CD80615DABF9102249D01C9639F31DCF
whonix.xz - 1,20 GB
656C3765AF80D6EF9AF892528811DCF39E5A37ED89073009D6788E2DB3C8E58A

single step decompression without temporary files:

7za x whonix.xz -so | srep -d -hash- - - | pzlib d - -o - | 7za x -ttar -si

on my PC, it takes 8 minutes to extract the gateway.ova and workstation.ova files with this command

1 Like

You can create a 1,15 GB file with FreeArc using -mpzlib+srep:m4f:hash-+dispack070+lzma:max:128mb, you can add --nodates

You need to add pzlib to arc.ini for this to work:

[External compressor:pzlib]
header = 0
packcmd   = pzlib e -v - -o - <stdin> <stdout>
unpackcmd = pzlib d - -o - <stdin> <stdout>
1 Like

Successfully extracted whonix.xz using 32 bit XP VM with 512 MB RAM

@Ego I would like to see your results

1 Like

Could the installer choose between 32 or 64 bit executables depending on the system? 64 bit was 20-30% faster on my tests

Whonix 14 will be 64 bit only.

I read that it is possible to run 64 bit VM under 32 bit OS as long as the hardware supports it

https://www.virtualbox.org/manual/ch03.html#intro-64bitguests

1 Like

@Ego any progress? do you need any help?

1 Like

Good day,

First of all, please excuse my limited activity on here, I sadly had to attend a few private matters lately which had quite some Impact on the amount of time I could dedicate to this.

Furthermore, the thing is that I’m not sure whether we as an open Project really are able to rely on Software wose source Code is sadly not publicly available. Telling someone who’d like to build from source that they should use a BugMeNot account is not only problematic from a Publicity perspective but also from our Basic philosophy of making Things openly accessible.

The developer fortunately has a Github account though this Project is sadly not on there. I’ve contacted him asking whether he could make the source Code available there or in another public form.

Again, it is hard for me to express how thankful I am for your Support. I am Aware that it can seem exhausting since I sadly have to go this route in order to be able to provide the source Code as is intended.

Thank you very much and have a great day,

Ego

1 Like

He has been online today on fileforums, have you received an answer?

In either case, I thought the best place to upload the files while not looking like claiming ownership of the project myself would be the other forum page he opened, which he did not update with the last version and source code. It could be downloaded directly from there without an account, thanks to this I also archived the page and the files on archive.org. Is this good enough whether/until the developer himself replies to you?

Forum page: https://encode.ru/threads/2506-pZLib
pzlib_1210.7z (645.8 KB) https://encode.ru/attachment.php?attachmentid=4848&d=1489555841
pZlib_source.7z (49.6 KB) https://encode.ru/attachment.php?attachmentid=4849&d=1489555841

On archive.org
Forum page: https://web.archive.org/web/20170315054347/https://encode.ru/threads/2506-pZLib
pzlib_1210.7z: https://web.archive.org/web/20170315054347/https://encode.ru/attachment.php?attachmentid=4848&d=1489555841
pZlib_source.7z: https://web.archive.org/web/20170315054347/https://encode.ru/attachment.php?attachmentid=4849&d=1489555841

SHA256 Hashes:
pzlib_1210.7z:
8BB4549DFD51274E86307A5F55F4E4073F50AC6E6EC729EDC2423B5E6AA58A0A
pZlib_source.7z: A9ED561D8F104B1A3217D8437C28F3CDA5BBB2FF80D30EF6124B5EAFEA59BBC3

1 Like

This is a huge opportunity, it would be sad if we don’t make use of pzlib, like srep they are state of the art tools, there is no similar tool that:

  • actually works
  • is open source
  • supports multi-threaded decompression
  • stdin-stdout support (extracting in a single step without temporary files and faster), very important and it is not very common
  • reproducible results
1 Like

Good day,

Currently, I am still awaiting a Response.

Furthermore, I fully agree with you considering the Statement that this would be massively beneficial. Either way, I am spending the time until this Situation resolves on implementation and testing (the part which eats time like nothing) so there is nothing lost.

The issue is that the pzlib source files do currently not contain a Copyright notice making any use on our site legally problematic, as the installer as a whole being under MIT would lead to me appropriating that Piece of Software. While I am strong in believing that the creator would have no issue with that seing the way he provides pzlib, that sadly doesn’t Change the legal and General Situation.

Thank you for your Support,

Ego

2 Likes

Greetings everyone, I’m Razor12911, the creator of pzlib. I’ve been going through this post for the last couple of hours, one thing that I’ve noticed is that you guys want to use what I have created, which is a zlib precompressor, well there really isn’t any issue its usage of the program but there is a condition utilising it, nothing major but the creator must be mentioned in the credits/acknowledgement list.

Regarding source, well unfortunately I cannot share the latest source however I noticed that my older source of the program has been linked to this thread, which is totally fine, it can be utilized without any problems license wise because the only acknowledgement to its creation goes to the creator of zlib and that’s it, everything else was self written.

Secondly what I noticed is you guys are planning on creating an installer for your product that must meet specific needs, well as it turns out, compression isn’t the only thing I specialize in, I can also create installation packages from top to bottom that utilize pzlib, srep and other compressors mentioned to this thread. I can help out if you want any custom installation creation, if you need any.

3 Likes

@Razor12911 Thank you for stopping by, I have a few questions, is the pZlib_source.7z file the source for this version: pzlib_1210.7z? It doesn’t include win64 object files, if we add them, would it compile the 64 bit binary as well? Do you think it is easy to compile the zlib object files from source? DelphiZLib 1.2.7 includes C++ Builder project files whereas 1.2.8 does not

2 Likes

Well you could get the already object files for x64 from here.

http://www.base2ti.com/

The reason I didn’t include them is because I’m a developer based on Delphi, anything other than that is just not my thing so the problem with x64 obj files was that they were 2 times slower than the x86 which is totally weird, but there is source as well in the attachment from the site, you can generate new obj files if you know how to do it because certainly, I have no clue how it’s done. I don’t they may not be slower for you but do check, I have been constantly changing obj files, it may or may not have been these ones that had this problem.

Goodluck.

1 Like

Good day,

First of all, thank you very much for joining our Forum and being willing to Support us in such a Major way. That is incredibly nice of you.

Furthermore, you may take a look at the current source Code and compilation methodology here and here.

Secondly, thank you very much for Clearing up the licensing question. Just for legal reasons, may I ask whether it would be ok to use your source Code, obviously with proper Attribution under the MIT license?

Thirdly, the x86 vs x64 Speed inconsistencies are likely caused by the requirements for extending the string lenght when running AMD64 based Systems in 64-Bit mode while the string lenght doesn’t fully utilize more then 32bits causing an increased requirment in using Cache which doesn’t yield any improvements while limiting the small amount of low Level Cache available on most consumer CPUs. That though is only a hypothesis. Maybe I simply misunderstood something my CS teacher told me, though that’s how I recall his reasoning on certain aspects of x64 being slower then x86.

Again, thank you very much for your help, if you have any questions or require anything, don’t hesitate to ask and have a great day,

Ego

2 Likes

As I know for Linux and Mac there is only 32bits versions and for Qubes only 64bits. For Windows what versions there are ?

Good day,

The installer includes both the 32 and 64bit version of VirtualBox. Whonix itself is currently being distributed in 32bit, which may run on any system, see also: 32-bit or 64-bit?

Have a nice day,

Ego

1 Like

Good day,

Adding to that, in the comming days, I shall release a completley new version of the Installer. It will be based on Inno Setup, increasing compatibility, decreasing size and optimizing stability. Furthermore, it has been designed in such a way as to make the compilation as easy as possible. After all the required files are in the same folder, you’ll only have to launch a batch file and everything will be handled automatically.

You may already compile it yourself: GitHub - EgoBits1/Whonix-Windows-Installer: Maintained, InnoSetup and 7-Zip based version of a simple automated installer, setting up Whonix, Whonix-UI and accompanying software.

Have a nice day,

Ego

1 Like

Are you sure,

  • 7zSD.sfx
  • 7za.exe

should be shipped since these are binary executable files?

Good day,

Problem is, the only way to currently utilize 7-Zip SFX in combination with a custom logo is by having these premodified via Resource Hacker. We can of course exclude them, though the installer would then carry the “7z” icon.

Have a nice day,

Ego

1 Like