kloak currently helps with obfuscating both keystrokes and mouse movements. The way kloak works is by recording evdev events as they are received from the input devices, then releasing them to the OS with jitter added to the timing. This is great for keystroke and mouse click events, but it works really poorly for mouse movement obfuscation for two reasons:
- Mouse movements generate an onslaught of evdev events. I get around 370 events per second from my MX Master 3S mouse. This quickly floods the queue and results in almost all events being delayed for the maximum amount of time allowed by the set kloak delay, getting rid of most of the jitter.
- Mouse event timing isn’t the only important thing to keep in mind when trying to anonymize mouse movements, the pattern of movements also needs to be obfuscated. Adding jitter to the timing won’t help with this at all.
Properly fixing this will likely require some algorithm that merges multiple mouse movement events into single events, thus resulting in the mouse pointer “teleporting” to the location the user tried to move it to, rather than the pointer moving smoothly. This however comes with its own unique problem - it will only work seamlessly in virtual machines. If you try to squish together mouse movement events and teleport the pointer on physical hardware, that erratic pointer movement will be very visible to the end-user and impair use of the machine. This isn’t a problem for Whonix per se since it (almost?) always runs in VMs, but for kloak in general it’s a problem.
There is a tool that gets around some of these problems called mouser (warning: have not audited code), which provides a virtual mouse cursor that the user can control the real cursor with. A left-click results in the real cursor teleporting to the clicked location, then clicking. Right click has similar behavior but with right-clicking. A middle click results in the real pointer moving but not clicking anything. This neatly resolves both of the above problems, but introduces some problems of its own:
- Middle-clicking is now impossible. The middle mouse button does exist for a reason, applications like Blender use it heavily.
- The user now has to operate the mouse cursor itself, rather than just operating the computer and using the mouse as a tool for that. There’s significant increased cognitive load in using the mouse in this scenario, which may be undesirable.
- Not all applications play nice with the virtual cursor technique used by mouser, as documented in the known issues. In particular, the cursor sometimes gets hidden, sometimes doesn’t show up in the right spots, and doesn’t work at all under Wayland.
- What even is the expected behavior when clicking and dragging? I can’t really think of an anonymity-preserving way to make click-and-drag work in this situation that isn’t awkward (combined left-mouse-button hold with middle clicking?)
Let’s assume we do go with a solution that involves combining mouse movement events in kloak. What algorithm should be used here? One technique might be to detect when a mouse movement event is coming immediately after another mouse movement event, and merge together the events as appropriate. This would probably be relatively simple to implement depending on the specifics of the math involved.
3 Likes
The implementation described at the end of this post will be a bit trickier than expected, since evdev mouse movement events can only represent one axis at a time (X or Y). This means that when a mouse event is received, it will take a bit of “digging” through the queue to find the event to merge it with. Mostly leaving this here as a note for myself.
3 Likes
I managed to implement a prototype version of this, and based on logs coming from my modified version of Kloak it seems to be working… almost. I ran into a complication I didn’t expect.
Obfuscating the timing of mouse events completely throws off relative pointer movement here, because of pointer acceleration profiles. Basically something in the stack (maybe in evdev? maybe in X?) doesn’t just look at the values of each evdev event, it also also looks at the time at which each event arrived. Events that arrive in rapid succession end up moving the pointer further than events that show up with long pauses in between. Implementing the “event merging” technique described above completely confuses the system, because now the actual events are much sparser (concealing the mouse’s actual path and event timing), and so Linux simply thinks the mouse isn’t moving all that much! The result is not just laggy, choppy pointer movement (expected), but extraordinarily slow, unusable pointer movement (not expected). Properly dealing with this will probably require figuring out the details of the acceleration profile algorithm in use, so that kloak can tweak the values so as to fool the algorithm into moving the pointer far enough.
Worst case scenario, we can abandon relative movement support, absolute pointer movement should be unaffected by this. But doing that means that if someone disables mouse integration in Whonix, they could be exposing biometric data. That’s obviously not desirable, so I think it’s better to start by finding the acceleration profile algorithm and figuring out how to work around it.
2 Likes
So, I did more debugging and discovered this isn’t actually a pointer acceleration issue like I thought, pointer acceleration wasn’t even enabled on my system. What was actually happening was my code was conflating EV_SYN -> SYN_REPORT events with EV_REL -> REL_X events. I’ve solved the problem now, and relative mouse handling now works as expected. The mouse is sufficiently well-behaved to be usable both with pointer acceleration disabled and enabled.
I did notice however that the mouse pointer movements are exhibiting a pattern I didn’t expect. Due to forum constraints I can’t post a video of the behavior, but because X and Y events are treated as distinct types of events, they end up being released at two noticeably different times. This means that the pointer oftentimes (indeed, most of the time) will teleport in exact straight lines either vertically or horizontally, as it moves towards the destination the user tried to move it to. If you move the mouse diagonally, the pointer will move in a stairstep pattern. This doesn’t affect the pointer’s accuracy (it will end up right where you tried to move it eventually), but it is a surprising effect, surprising enough that it might cause users to reveal different kinds of biometrics in the actions that users take to adjust for the strange behavior (this is similar to the covert impairments trick). That being said, even if that happens, it seems likely to me that the timing obfuscation will introduce too much noise to extract behavioral information easily, though I can’t prove this.
While this effect is most noticeable when using relative mouse movement, it can also be seen when using absolute mouse movement in a VM. If all you’re doing is moving the mouse and clicking, all of the erratic movement is hidden, but when you drag a window, the straight-line jumps and stairstep pattern will be clearly visible. It looks buggy even though it’s not.
Will try to see if i can get the X and Y events to be scheduled to be released at the same time, so that this jumpiness will be replaced with just lag. Lag is acceptable, this weird jumpiness probably isn’t.
2 Likes
Alright, that didn’t take long, now the stairstepping is gone 
As expected, the pointer is laggy, but that’s the best that we can get with this approach.
I’ve now pushed my changes to GitHub - ArrayBolt3/kloak at arraybolt3/anon-mouse. This should be usable in Whonix immediately, assuming this doesn’t introduce new bugs. There’s still more work that could be done to polish this though:
- kloak would benefit greatly from a key combo similar to the rescue key that would simply toggle event buffering on and off without terminating the background process. This “passthrough” mode would let people disable kloak momentarily so they can do something that requires precision or where input biometrics are desirable, then turn it back on when they want privacy back and can afford kloak’s less desirable implications (mouse and keyboard lag).
- Touchpad-driven pointer movement is NOT anonymized by the new algorithm, the old algorithm still applies there. This is because touchpads don’t generate normal relative or absolute movement events, but instead generate multitouch events. These are much more complex than normal mouse movements, and will require further research to properly handle.
2 Likes
So, touchpad support ended up looking to be a lot more of a mess than I thought.
- Linux’s multi-touch protocol is very complicated, figuring out what events to merge and how exactly to merge them will probably be pretty tricky.
- Touchpads are approximately as difficult to handle properly as timezones. The primary libinput author wrote a very good rant explaining the problems: Who-T: Why it's not a good idea to handle evdev directly
- One might think that a good solution to this issue might be to just strip any events that are “troublesome” and leave only those events that one may legitimately want to deal with. I tried this, it works… on my machine, and even then only to some degree. Touchpad movement works for the most part (it’s rendered choppy of course, and the start of the movement seems to be ignored), multi-finger taps work sometimes but not others, two-finger scrolling seems to still be functional, and physical button clicking works too. The fact that the above works on my machine does NOT mean it will work on anyone else’s machine, and even on my machine multi-finger taps are broken.
- I haven’t established how to get a touchpad to just pretend to be a mouse yet (generating only basic EV_REL events). That’s probably our only hope of a good solution at this point, and I don’t know if Linux even supports doing that.
2 Likes
Brain dump of the current state of the project, both for my own understanding and for transparency about progress.
Me and Patrick talked over the current progress made on mouse obfuscation, and decided that mouse lag is a severe impediment to using the system, and not something that is acceptable in a final implementation. However, avoiding mouse lag seems impossible initially, because without mouse lag, you can’t buffer and then time the release of mouse events, and if you can’t do that, you can’t anonymize mouse activity. To mitigate this, we had the idea of using the “double cursor” technique mouser uses, but slightly different:
- The user uses the mouse exactly the way they normally would. A “virtual cursor” shows the user where their mouse actually is, this cursor moves exactly like the mouse normally would.
- As the virtual cursor moves, the real mouse cursor (the one the rest of userland sees) will “chase” the cursor. The movement of the real cursor will have anonymization jitter added to it, so it will be laggy, but it won’t matter much because the user will be able to use their mouse normally otherwise.
- All mouse events are similarly delayed and have jitter added to them, but because of the virtual cursor you can always see exactly where your click is going to register once the pointer gets there and clicks. This should work transparently with other click-like events like middle clicking, scrolling, click-and-drag, etc.
Sounds simple enough, but as it turns out this is ridiculously challenging:
- As explained above, handling evdev events directly is a very bad idea. To work around this, libinput will have to be used to process the events coming from the user’s pointing device. This of course will add some significant complexity to kloak, though hopefully not too bad.
- libinput provides its own event stream separate from evdev, so that means its events will have to be handled in their own queue and then passed to the display server directly.
- Passing things to a display server directly means input emulation, similar to Synergy or remote desktop software. This is where things get really sticky, because now we get to deal with the glorious fragmentation that is X11 vs the Wayland ecosystem (which ecosystem is itself horribly fragmented).
- Under X11, the proper way to emulate input is to use the XTEST framework. This works pretty much universally in the X11 world.
- Under Wayland, you’re apparently supposed to use libei, which honestly doesn’t look that bad. Where things get fun is that, if kloak goes this direction, it has to support both XTEST and libei, which of course is not exactly fun. I did some web searching and couldn’t find a utility that would allow the user of libei with X11, though ironically I did find out that apparently an X11 client can use XTEST with XWayland, and that will talk to libei which will then talk to the Wayland compositor. So I guess an XTEST-only implementation would be an option.
- We also need to draw the virtual cursor. Here again X11 and Wayland take two different approaches:
- Under X11, you basically make a transparent fullscreen window and draw to it. Simple enough in theory.
- Under Wayland, the window isolation features that give Wayland better security become a serious pain - there is no way to tell the server to draw something “directly” on the screen, and you can’t pop open a fullscreen window. What you can do is use the wlr layer shell protocol which will let you do effectively the same thing as under X11 (make a big transparent overlay), but GNOME at least doesn’t support the wlr layer shell protocol, and probably never will. For Whonix in particular, this would probably be an acceptable solution, because we’re intending on using labwc, which is wlroots-based and will support this protocol, but still, pain.
- One might ask, “can’t you avoid all of this and just feed events back into the system via evdev?” On the surface that might sound OK, since we can emulate a touchscreen with evdev and generate evdev events that are sensible, however this runs into a roadblock with multi-screen setups (which are rather common and can’t be overlooked). evdev doesn’t provide any way to say which input device corresponds to which screen, that’s for the display server to figure out. In my experience with Kubuntu 20.04 back in the day, X11 will interpret a single touchscreen as somehow stretching across all displays on the system, I don’t know how Wayland will handle the same situation. At any rate, making one “touchscreen” per display isn’t really going to work unless we want to reconfigure the display server to properly bind each virtual touchscreen to each physical display. This is doable under X11 maybe, it might not be possible under Wayland in a compositor-independent way.
- It might seem like you could make a single virtual touchscreen that stretches across all displays, but this breaks down because evdev doesn’t understand the concept of pixels for you to place the mouse pointer at. It understands device-specific “units”, and can map those units to a physical distance via some resolution data. The display server then takes that data and does fancy calculations on it to get a pixel position. We could just figure out the smallest possible “touchscreen” size in pixels that maps to all displays, then make our virtual touchscreen that size in units, but then we’d need to also figure out DPI so that the display server will know how to translate our virtual device’s units into pixels. If all the displays on the system are run-of-the-mill 72dpi devices, this is fine, but as soon as you throw a HiDPI display into the mix, you now have to figure out the right resolution to use, you can’t just hardcode it or guess it. Even that could be lived with in theory, but then you have people who use a mixture of HiDPI displays and typical 72dpi displays, at which point there is no right answer for what DPI to make your virtual device. At this point I guess you could make the virtual touchscreen’s DPI equivalent to the densest display’s DPI, then multiply the pixel values from the lower-DPI displays as appropriate to map the virtual touchscreen to them properly, but to me I feel like we’re begging for off-by-one errors if we do that, and there may be something deep in the guts of one of these components that will throw an utter tantrum when used in this way, so… I’m not really too keen on this approach.
- In any event, we’re capturing and doing involved computations on mouse data, so if we want any nice features like mouse acceleration or the like, we get to implement those in kloak itself with this approach.
- Depending on how things work, there could be some risk of event reordering between the keyboard and mouse, and if that happens then we’ll have to use libinput and XTEST (and libei?) to handle keyboard events too, so that all the events can be in one queue.
No matter what direction is taken for doing this, things quickly become pretty complex, just to draw a red box where the mouse pointer will eventually end up and handle touchpads right. I far prefer how Qubes works in this regard, where we can work with X11 events directly. That’s nice because there’s no bizarre hardware behaviors to take into account like when dealing with evdev, and there’s only one display server implementation to worry about. Unfortunately I already looked for ways to intercept X11 input events before they reached the server to manipulate them, and it doesn’t appear that such a thing exist. The situation is similar with Wayland as I understand it, Wayland uses libinput which uses evdev and there’s no way to insert own code between the Wayland and libinput layers, short of forking a Wayland compositor which we are NOT about to do. If you want to effectively do that, you have to use libinput yourself and then use emulated input, and, well, that gets you to where I got above.
2 Likes
Hello, i am a mouser developer, i am so interested in your development, thanks for sharing your development process. Unfortunately idk C/C++ and cannot help to you. Python is so limited for tool like that. I hope you will develop it soon, thanks anyway. 
This can be simplified a bit by having kloak support Wayland only.
Once Whonix gets ported to Debian trixie, we’ll use Wayland anyhow. Therefore no need to take on the challenge of X11 support.
1 Like