AI models increasingly powerful. Capable of finding 0 days and escaping latest KVM (Trail of Bits). Opinion: AI usage necessary part of security.
Does Whonix project use AI to help security? Scans with best models? Joined cyber access tiers of AI companies (Daybreak/Glasswing)?
Read wiki Whonix Policy On Artificial Intelligence (AI), not clear on how it is used.
Yes, quite heavily. Enough so that at least one person thought (incorrectly, thankfully) that we had become a vibecoded project:
We have a pretty sophisticated set of both human and automated safeguards so that we can use AI to find security issues and harden our code while avoiding the risks that usually come with vibecoding. (We’re quite enthusiastic about AI’s capabilities, but also treat it as an external contributor, meaning we generally assume its output is vulnerable and/or malicious until proven otherwise.)
We use the best we have access to. At the moment I believe that’s a slightly older version of Claude Opus (we tried newer versions but they didn’t give us as good of results).
I am not aware of the project having done this. I doubt we’d be able to get access to those kinds of tools due to our somewhat niche status. That being said, new models are coming out that are very good at finding vulns while being not-so-good at exploiting them, and those are becoming easier to access, so hopefully we’ll adopt them once we can get to them.
That’s an overview of how contributors (maintainers included) are to use AI in their work. It’s basically “you don’t have to use AI, but if you do, here’s what we expect from you.”
Should apply. Likely Whonix or contributors will be accepted. Friends have OpenAI Daybreak Blue / Anthropic Cyber Verification Program. Friends not famous people, working on projects with less reach than Whonix. Minimal verification needed, think only ID/face. Likely individual contributors accepted.
Know some companies have Daybreak Red, think access requires organization application. Possibly accepted. Glasswing likely impossible.
More power, less blocks than normal served models. Strongly suggest.
My experience: latest best models find more subtle bugs than predecessors. Friends experience same. Benchmarks align. Still impactful bugs despite subtlety. True for GPT-Astra and Fable 5.1 to me. Better result, even when not verified in previous mentioned programs. Opinion: worth using Astra/Fable to find bugs. Worst case, Kimi K3 does not block for security queries. Heard better than Opus but never tried.
I seriously doubt Daybreak Red would be at all useful to us, since we’re purely on the defensive security side of the malware arms race. Blue might be interesting theoretically.
Worst case scenario, if a community member with access to fancy models wants to scan Whonix, they’re welcome to do so (you included if you have that kind of access). If a model finds anything interesting, please follow the security vulnerability reporting process:
Big software companies use red teams. Attack = finding bugs, I think good. Blue/CVP worth trying, likely you will be accepted.
Would try when I have time. I don’t have program access. Don’t want to give up privacy. Since you can get access, likely better result if you also do it.
From what I understand, the Blue models are good at finding vulns but less good at exploiting them. Red is good at finding vulns and weaponizing them. For our own code, finding the vuln is enough, we know the code well enough to know if something is a problem even if we don’t have a working exploit for it. Weaponizing a vulnerability might be useful in theory to prove to a third-party project that a bug is really a problem, but the few times we’ve needed to do that, we’ve had the sufficient skill to weaponize a vuln ourselves so we could include a sample exploit in a vuln report. Even if we had access to Daybreak Red, I doubt we’d be scanning other people’s code with it. (Plus, Daybreak Red is approximately three times as expensive as Daybreak Blue.)
Yes. Sometimes Blue users complain of blocks in finding stage. Red marginally more useful still.
Would reassure to know Whonix is using best cybersecurity models and joining important available programs. Strongly suggest.
Unfortunately cannot do myself since no program access. Privacy reasons. But will try with public versions later.
7 posts were split to a new topic: Vulnerability bug reports by viajred [AI assisted]
2 posts were split to a new topic: Whonix-Gateway Firewall /usr/libexec/whonix-firewall/enable-firewall firewall_lockdown function nftables non-atomicity security report [AI assisted]
What should be the scope of this? Review Whonix’s (and Kicksecure’s) own source code?
At this time, AI model quality doesn’t seem to be the limiting factor. I will elaborate at length, why I think that is so.
Opinionated. Some anecdotes based on personal experience.
For now, I stick with Claude Opus 4.8. I know, on paper Claude Mythos (or Fabel) is supposed to be better. But for the things I am working on, the newer models turned out to be actually worse.
Paraphrased, not verbatim Claude quotes:
This is cybersecurity work. Refusing request. Redirecting to Claude Opus 4.8.
It was also more lazy.
Done this and that. Checkpoint reached. Continue? (Despite
/goal.)
This session is going quite long already. Suggest to open a new session.
The weekly quota provided by Antropic for the Claude Max 20x subscription plan (the highest available subscription at the time of writing) that I am using is also exhausted pretty quickly.
Then the news broke “Claude Mythos” suspended for Non-U.S. citizens due to “national security”. The U.S. did believe the “our ai model is so good, it even endangers national security” marketing. As if there are no U.S. citizens who allow a foreigner access to their account. Nowadays those national security concerns have been addressed at least somewhat somehow.
While the Fabel model was available, I didn’t notice any higher quality. Only approximately twice the speed of quota consumption.
Then an option to apply for access to cybersecurity models was made available. (Anthropic Cyber Verification Program (CVP)) Within 2 days, the decision was “no”, with no reasons given. Based on the non-responsiveness / response timing of Claude support generally, I would speculate that the application processing is also AI based and the AI decided “no”.
AI benchmarks: Problematic. [1] [2] [3] It’s difficult since replies by AI are randomized, non-deterministic. Are those AI slop or sensible? Did AI companies train on the test to archive better test results? When benchmark results carry immense commercial value, how do we know maintainers would reject an undisclosed offer to manipulate them?
I am not someone who keeps over-optimizing the tooling while losing focus on the big picture getting excited about every week’s news of some new AI model or AI trick despite diminishing returns.
The amount of time and expenses that can be spent on AI are unbounded.
dist-ai is approaching 1000 AI-generated tests. New tests are added almost every day. So yeah, AI is heavily used for AI based security review, but I haven’t nearly exhausted even using standard models.
Related:
[1]
[2]
[3]
Unfortunate. Still, think it is worthwhile to try OpenAI models. And apply to Daybreak Blue. Less strict than Anthropic. Trail of Bits KVM escape done by GPT 5.6 Cyber. Will explain.
AI benchmarks: Problematic. [1] [2] [3] It’s difficult since replies by AI are randomized, non-deterministic. Are those AI slop or sensible? Did AI companies train on the test to archive better test results? When benchmark results carry immense commercial value, how do we know maintainers would reject an undisclosed offer to manipulate them?
Subjective, I can notice improvement each generation. New bugs not found by previous generations found. (Trail of Bits used 5.6 Sol subjectively better to Opus 4.8.) Objective:
From Kicksecure own policy: Kicksecure Vulnerability Disclosure Policy . Quotes Linux Kernel maintainers.
If you resorted to AI assistance to identify a bug, you must treat it as public. While you may have valid reasons to believe it is not, the security team_s experience shows that bugs discovered this way systematically surface simultaneously across multiple researchers, often on the same day. In this case, do not publicly share a reproducer, as this could cause unintended harm; just mention that one is available and maintainers might ask for it privately if they need it. Security bugs — The Linux Kernel documentation
Bugs discovered simultaneously. Often on the same day. Implication. When new best model released, users rush to run security scans. Previous bugs not surfaced by human or previous model instantly visible. (Same day implies day of model release. Otherwise would trickle in. “Same day 8 days after model release” scenario unlikely.)
Malicious users also run scans. Therefore, can immediately use bugs.
Therefore, immediate adoption of new best models important.
You implemented fix for my last security report. Issue discovered by GLM-5.3, best public weight model. Only Workstation root malware was required to induce Gateway to send clearnet packet to malware chosen destination. High impact.
I hope it is sufficient proof of things I said? If insufficient, what can convince you? Is your belief falsifiable?
The amount of time and expenses that can be spent on (anything) are unbounded. Strong results obtained with reasonably low time/expense. I recognize Anthropic denied you CVP, models refuse requests. I can donate to cover OpenAI subscription expenses. If needed. 100-200 US$ plans should be sufficient.
I will try reviews with OpenAI models without Daybreak. Expect to be blocked. Must restate AI review should be continuous process, not one time. New best model, run new review. Joining cyber programs increases speed of access to new best models.
Note that the pricing of Daybreak is billed per token, not per month, see:
Also note that we are looking into how to get access to ChatGPT Daybreak internally. We still don’t know if it’s possible, but we’ll find out.
Seems link only quotes API price, does not say it cannot be used with monthly subscription? -cyber suffix models are Red. Think Blue applies to all models. Pricing is same. Likely valid via subscription. See: https://help.openai.com/en/articles/20001259-openai-daybreak-common-issues-and-troubleshooting#gpt-6-sol-gpt-6-luna-and-astra-availability
GPT-6 Sol and GPT-6 Luna support reduced refusals with Daybreak Blue when your account is approved and Daybreak is enabled for the request.
Astra keeps standard safeguards under Daybreak Blue. Reduced refusals on these models require Daybreak Red access.
(Shows Red is better because of Astra reduced refusals)
Know some individual Blue users. Think they would not pay API pricing. Think only Red/Organization/Enterprise might be forced to API pricing. Seems even 20 US$ subscription can use Blue. See: Did losing ChatGPT Plus cause my Daybreak Blue access to be revoked? - Bugs - OpenAI Developer Community
Even if true (unlikely), API pricing not excessive. GLM-5.3 bug you fixed cost 150 US$. Paid API cost. Sol only around double cost of GLM-5.3. Can still cover API costs. If you are lucky to get Red, Astra still likely worthy.
Hmm, maybe I misunderstood, OpenAI does seem to be confusing here. I guess we’ll find out 
There are dozens of AIs claiming to give more freedom and capabilities about exploits and vulnerabilities here are couple of exaples but i dont know if they are genuine or fake or trash results.. (didnt try, not much of a fan of AI) but here is a list: (dont pay, just read)
And some websites give services for easy deploy and build models immediately e.g:
I like the idea of msf + armitage back in the days in kali which can be now done more into an AI model but based on the experience from big known AIs like GPT,Grok,Gemini.. AI still a trash compared to a genius hacker finding and exploiting stuff manually.
As I said, I haven’t exhausted public-accessible-by-most-people-that-pay models yet. There are a lot of lower hanging fruit to pick. I am following a structured approach.
A pure AI chatbot cannot “just scan Whonix”. Pure chatbots can only theorize. They’ll theorize, not verify. And they produce so much text, that it’s too much to verify for humans.
To be really productive, the AI needs “hands”, local code execution rights (“tool calls”), i.e. a harness. On top of that it needs infrastructure, such as, it needs a machine with root access (because leak test applications often require root), the Whonix VMs must be installed, the harness must be able to execute code inside Whonix-Workstation with root rights, automate boot into sysmaint session and then verify (not theorize) findings by running the leak testing software on the host operating system or local LAN proxy server.
Building that infrastructure can be done using vibecoding on an untrusted server, but it requires a large amount of prompting, which I am doing. This perquisite work doesn’t need any top models yet. Once infrastructure is ready, more or less. Now I am in process of automating leak tests. Not a as one-off “try to find a leak” but a vibecoded, untrusted, comprehensive, ai driven automated leak test suite.
This will be presumably be more comprehensive and toke efficient (lower expenses) in the long run as the test suite can ideally be re-run for every new release, every upgrade.
Once done, the automated leak test suite will be ai reviewed by claude (openai), codex (chatgpt), agy (google), grok (xAI) for correctness issues and testing gaps. Once that is done, top AI models - if access can be acquired - can check for correctness issues and testing gaps or roam free.
Robust and good design.
Optimal. Consider: Even without this, harness on machine with root can read code statically, find important bugs. User can run full POC manually on real Whonix. Viable with best models. Best models have fewer false positives. Can eliminate FP statically.
Therefore, if optimal Whonix VM control harness not yet complete, performing static procedure holds value. If best cyber models approved before harness complete, consider static run.
Subjective, I experience “roam free”, “find the bug which does <impact>” finds subtle, high impact bugs. Seems model treats writing big test suites as chore, roam free as challenge.