When I use the nyx monitor in gateway, my ram usage get bigger and bigger until gateway freeze
xfce 16.0.8.2
nyx 2.1.0 jan 12, 2019
Generic Bug Reproduction required.
Some discussion on memory leak of nyx on GitHub
Can reproduce a memory leak of nyx on Whonix-Gatway (version 17). Tor version 0.4.7.16, nyx 2.1.0-2.2 and stem 1.8.1 (all versions included with Whonix-Gateway).
To reproduce: Have nyx running on Whonix-Gateway and on Whonix-Workstation open many links in Tor Browser across separate domains to generate many streams (The various links on the Whonix welcome page are good for this) and observe memory usage ballooning. Notice how even when Tor Browser is closed and Whonix-Workstation is shut down the memory usage of nyx does not go down.
Unfortunately nyx is unmaintained upstream so distribution of any improvements to it would have to be coordinated by the Whonix project if desired.
Thank you.
I don’t plan on software forking nyx. If someone wants to contribute to nyx, please work with the The Tor Project.
Quote atagar from Needs more than two gigabyte RAM if running for two days · Issue #52 · torproject/nyx · GitHub
I’m the author of both Stem and Nyx. I left Tor a couple years ago whereupon Georg (gk at torproject dot org) took over but he doesn’t monitor GitHub. You’ll need to email him if you’d like to get his attention (see this issue for the latest discussion on that). You can also try tor-dev@.
Try this if you want to see this fixed.
This was not a final plan or demand for action. Just some thinking for the issue.
If the upstream Tor project is willing to collaborate towards either investigating or maintaining a solution that would be ideal.
Next challenge is to investigate the memory leak itself and what would be needed to fix it. Some investigation has been conducted in the above linked GitHub issue but is not conclusive. For example the version of stem included with Whonix-Gateway and was used for my test is 1.8.1 while there has been another relase since them.
Thank yo
Whonix ships the version that comes from packages.debian.org. Related: Operating System Software and Updates chapter Frozen Packages in Kicksecure wiki
(Whonix is based on Kicksecure.)
My current thinking at this stage is to reproduce this memory leak with latest nyx and stem from upstream to remove potential variables under question. If the root cause (and potential solution) can be identified that would be best. From there perhaps current Tor developers could be engaged in this discussion.
Thank you.
Conducted another test in more detail.
nyx (commit dcaddf2) and stem (commit 2074fdee) were installed into a virtualenv on Whonix-Gateway for testing.
Afterwards traffic that comprised of many discrete circuits was generated on Whonix-Workstation and the memory usage of the nyx process was recorded. Afterwards Whonix-Workstation was shut down and NEWNYM was sent to the Tor control-port. The memory usage did not decrease substantially from the maximum.
Memory usage of nyx before stress-test. (ps aux)
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
user 5729 5.9 4.8 644476 47636 pts/0 Sl+ 13:48 0:00 python ./run_nyx
Memory usage of nyx after stress-test.
user 5729 1.8 41.4 996824 407392 pts/0 Sl+ 13:48 0:19 python ./run_nyx
Verdict. The memory leak issue still exists when tested with upstream version of relevant utilities.
In addendum the onioncircuits utility (from Tails and as included with Whonix-Gateway) was tested and it does not exhibit the same issue.
Try reporting this at the Tor project issue tracker.
There is already a discussion on GitHub for this issue.
nyx is not listed as a project on gitlab.torproject.org so I wonder if they are interested in it or not.
To Patrick: I sent you an e-mail from the address registered to this account.
Not warranted in this case as per Private Communications Policy.
Certain aspects of intra project diplomacy are best not conducted in a public forum but I will summarize the situation regardless.
-
Issue has been confirmed with current versions of libraries under Debian/Whonix-Gateway
-
Reported these recent findings on the GitHub issue for the benefit of others as well.
-
Original developer of nyx/stem is no longer involved with Tor.
-
Attempting to contact at least one maintainer from Tor Project with these findings has not elicited a response.
Having said that I might still investigate what is leaking memory with nyx in the meantime. In addition if anyone else has any suggestions that would be appreciated.
1 Like
Unfortunately happens often that projects are abandoned, no longer maintained, no responses. This goes for any project not only the ones discussed here. Static that isn’t controversial. Tons of bugs did not get responses.
Left to do: Open a ticket a Tor Project issue tracker.
Yes that was my thinking too. However nyx as a project does not appear to have an entry on the Tor Project GitLab instance and I am unsure if they have a ‘catchall’ issue-tracker for projects not explicitly hosted there.
In any case I may conduct further investigation on my own and would be open to collaborating with others if they would be interested.
h3xagonal via Whonix Forum:
Yes that was my thinking too. However nyx as a project does not appear to have an entry on the Tor Project GitLab instance and I am unsure if they have a ‘catchall’ issue-tracker for projects not explicitly hosted there.
Then I’d suggest to just open a ticket against any component sounding
like sysadmin, gitlab or something and tell them there’s no tracker for
nyx yet.
The main point is to start communication, opening tickets in public and
see the (non-)reaction.