Whonixcheck, timesync GUI, msgcollector

Development
1

Yeah. Now that we have a python coder, I am really excited! That would have to be added to GitHub - Kicksecure/msgcollector: GUI / CLI Toolkit Library - collects messages send to it by applications (API) and dispatches them when instructed to do so by the application [message notification system developed for Whonix, that can show messages in X as well as in terminal, usable by anyone, not just Whonix, soon]. I package it a bit better soon. Nevertheless, a python GUI could be called from https://github.com/Whonix/msgcollector/blob/master/usr/lib/msgcollector/msgdispatcher.
Not required to understand full msgcollector (certainly helpful, but not mandatory for this). When you search that script for zenity, that’s the part that would have to be changed. Instead of running zenity with all those arguments, a python script could be called. Also instead of running kdialog (for passive popups) we could also consider using python if that would bring any advantages.

2

Done.

3

Working on msgdispatcher popups.

Is there a way to reload msgcollector/msgdispatcher without restarting the VM?

4

Less important background information in [].

To stop.

To start.

[Emulate /etc/xdg/autostart/msgdispatcher.desktop.]

Just run.

[Which doesn’t do much more than “/usr/lib/whonix/msgdispatcher --x”.]

5

For completeness sake. To check if running or not.

(Useful for a manual check. Not an automated one. For an automated one I’d check “ps -p $pid”'s exit code.)

6

Some progress on the python popups (zenity replacement only). Not easy to navigate in the different scripts involved by msgcollector/msgdispatcher.

I have some trouble trying to get the value of icon ($icon) in msgdispatcher, in order to display the proper message icon (information, warning or critical).

On a larger point of view, in bash, where do you assign a value to the variable ‘msg’, for example? In msgdispatcher, the only place is at the beginning of the scripts, the rest are references to its content ($msg). Is ‘msg’, or ‘title’, or whatever, a system wide variable? If so, how is that possible? I am still in wonder when I read a shell script.

7

msgcollector isn’t very relevant here. Well, depending on our goals. In essence msgcollector is gets run by whonixcheck etc. and stores information (such as message type, message, etc.) in text files. When whonixcheck says “it’s done”, it tells msgcollector, which will create a .done file.

msgdispatcher gets started in CLI (by /etc/profile.d) and X as a background progress that watches for .done files. As soon one exists, it reads the other information (message type, message, etc.) and dispatches the message using zentiy.

I created that message notification system, because non existed that would allow background processes to write messages to X as well as to CLI. For performance reasons, timesync and whonixcheck start when able to (using the dependency based init.d system). In other situations, like with NTP and other daemons, users would never know it it failed, since they only write failures into their log. Also such background processes usually have no way to send messages to CLI. In general, in Linux systems there was no way to keep users who are using CLI posted about important status messages. Also such background processes usually have no way to send messages to X. In Whonix’s case, network time synchronization is important for anonymity reasons. When it’s still in progress, failed or succeeded are information the user should be aware about. Also without msgcollector there is no simple “add this to X AND CLI output”.

Due to absence of python skills during development, I had to work around limitations of zenity. Whonixcheck / timesync gather lots of bits of information. But once a zenity window opened, at least as far I know after extensive research, content of the window can be no longer changed. And a million zenity popups would be annoying. So the idea was to collect messages and to show one when it’s done. msgcollector provides a nice interface for shell scripts to interact with CLI and X output messages including features such as “show progress bar only if needed”.

I guess only replacing the zenity part with python would be the simplest. Make a much more pretty popup. Depending on moot, the whole msgcollector system could be rewritten some day in python. Or as a partial rewrite, other nice things such as filling up the whonixcheck result window as results come in could be doable.

I have some trouble trying to get the value of icon ($icon) in msgdispatcher, in order to display the proper message icon (information, warning or critical).
That is a bug. Sorry about that. In Whonix 8, the icon variable is empty. Hence zentiy never shows a small icon on the left side of the window title (favicon). in Whonix 9 the following code will be used to set the icon variable. 
   if [ -f "/var/run/msgcollector/${identifier}.icon" ]; then
      local icon
      icon="$(cat "/var/run/msgcollector/${identifier}.icon")"
   else
      ## Fallback.
      local icon
      icon="/usr/share/icons/anon-icon-pack/whonix.ico"
   fi

In meanwhile for your tests in Whonix 8, I suggest to add a line.

icon="/usr/share/whonix/icons/whonix.ico"

Right above the zenity line.

On a larger point of view, in bash, where do you assign a value to the variable 'msg', for example?
Search msgdispatcher script for. 
msg=

Then you’ll find it. In essence it’s read from a file, that was previously created by msgcollector. But you don’t have to worry about that. When you replace the zenity command with ‘/some/python/script --msg “$msg” --type “$type”’, then these variables will be already set. They are already available to you.

Is 'msg', or 'title', or whatever, a system wide variable?
No.
If so, how is that possible?
It's not possible.
I am still in wonder when I read a shell script.
Same for me when I look at python. :) Makes me hope that it's not that difficult after all.
8
I guess only replacing the zenity part with python would be the simplest. Make a much more pretty popup. Depending on moot, the whole msgcollector system could be rewritten some day in python. Or as a partial rewrite, other nice things such as filling up the whonixcheck result window as results come in could be doable.

For the moment, zenity is the priority, to get rid of the window resizing outside the screen space. Filling up the whonixcheck result window as results come in would be very nice. I don’t think I have seen it before, graphically.

That is a bug. Sorry about that. In Whonix 8, the icon variable is empty. Hence zentiy never shows a small icon on the left side of the window title (favicon). in Whonix 9 the following code will be used to set the icon variable.

I’ll check on that. Still not crystal clear to me.

Then you'll find it. In essence it's read from a file, that was previously created by msgcollector. But you don't have to worry about that. When you replace the zenity command with '/some/python/script --msg "$msg" --type "$type"', then these variables will be already set. They are already available to you.

Yes, the script is working, except for the icon. For the moment, there is only an information icon. It looks prettier, using the system font, and the size of the window is automatically adjusted to the text.

Same for me when I look at python. :) Makes me hope that it's not that difficult after all.

I’m surprised you never gave it a try. With PyQt or PyGtk, you have to deal with objects and events, but the core python seems largely affordable, I think, especially when you master bash scripting.

9

The preliminary version of the python script replacing zenity in msgdispatcher is ready.

I could not clone https://github.com/Whonix/msgcollector/blob/master/usr/lib/msgcollector/msgdispatcher.

error: The requested URL returned error: 403 while accessing https://github.com/Whonix/msgcollector/blob/master/usr/lib/msgcollector/msgdispatcher/info/refs

So, here are the modifications.

In msgdispatcher.
error-handler()

error_handler() {
   local exit_code="$?"

   local msg="\
###############################################################################
## $scriptname script bug.
## No panic. Nothing is broken. Just some rare condition has been hit.
## Try again later. There is likely a solution for this problem.
## Please see Whonix News, Whonix Blog and Whonix User Help Forum.
## Please report this bug!
##
## BASH_COMMAND: $BASH_COMMAND
## exit_code: $exit_code
###############################################################################\
"
   echo "$msg"
   mkdir --parents ~/".whonix"
   echo "$scriptname: BASH_COMMAND: $BASH_COMMAND | exit_code: $exit_code" > ~/".whonix/msgdispatcher-error.log"
   #####################################################
   ## Should pop a graphical message.
   /usr/lib/whonix/dispatch_X.py "warning" "$scriptname" "$msg" 
   #####################################################
   true
}

dispatch_x_active()

[code]dispatch_x_active() {
trap “error_handler” ERR

Pyhton script replacing the zenity call below.

/usr/lib/whonix/dispatch_X.py “$type” “$title” “$msg”

#if [ “$display” = “” ]; then

display=“:0”

#fi

#if [ -f “/var/run/msgcollector/${identifier}.lefttop” ]; then

local lefttop

lefttop=“1”

#fi

“sudo -u user” is required, because /etc/init.d/sdwdate runs

as root. When making changes, test this using:

sudo touch /var/lib/whonix/sdw_error

sudo service sdwdate restart

#sudo -u user zenity \

–display=“$display” \

–height=600 \

–width=800 \

–timeout 86400 \

–no-markup \

–window-icon “$icon” \

–title=“$title” \

“–$type” \

–text “$msg” \

>/dev/null 2>/dev/null &

#echo “$msg”

#if [ “$lefttop” = “1” ]; then
## provided by /usr/lib/whonix/msgwmctrl

output_wmctrl_move_window_to_left_top “$title”

  ## Not in Debian Wheezy, due to a bug in zenity resulting in no visible
  ## text. Can be re-enabled in Debian Jessie.
  #output_wmctrl_maximize_window "$title"

#fi
}
[/code]

New script /usr/lib/whonix/dispatch_X.py

#!/usr/bin/python
# -*- coding: utf-8 -*-

import sys
from PyQt4 import QtGui


class dispatch(QtGui.QMessageBox):
    
    def __init__(self):
        super(dispatch, self).__init__()
        
        self.initUI()
        
    def initUI(self):               
        
	## To be done: set maximum size, scrollbar if text too long
        #self.setMinimumHeight(0)
        #self.setMaximumHeight(768)
        #self.setMinimumWidth(0)
        #self.setMaximumWidth(1024)
        #self.setSizePolicy(QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Expanding)

        if str(sys.argv[1]) == "info":
            self.reply = QtGui.QMessageBox.information(self, str(sys.argv[2]) , str(sys.argv[3]),    
            QtGui.QMessageBox.Ok, QtGui.QMessageBox.Ok)
            sys.exit(0)
        elif str(sys.argv[1]) == "warning":
            self.reply = QtGui.QMessageBox.warning(self, str(sys.argv[2]), str(sys.argv[3]),  
            QtGui.QMessageBox.Ok, QtGui.QMessageBox.Ok)
            sys.exit(0)
        elif str(sys.argv[1]) == "critcal":    ## This one puzzled me.
            self.reply = QtGui.QMessageBox.critical(self, str(sys.argv[2]), str(sys.argv[3]),   
            QtGui.QMessageBox.Ok, QtGui.QMessageBox.Ok)
            sys.exit(0)
        else:
            sys.exit(1)


def main():
    
    app = QtGui.QApplication(sys.argv)
    ex = dispatch()
    sys.exit(app.exec_())


if __name__ == '__main__':
    main()
10

[quote=“troubadour, post:9, topic:246”]The preliminary version of the python script replacing zenity in msgdispatcher is ready.

I could not clone https://github.com/Whonix/msgcollector/blob/master/usr/lib/msgcollector/msgdispatcher.

error: The requested URL returned error: 403 while accessing https://github.com/Whonix/msgcollector/blob/master/usr/lib/msgcollector/msgdispatcher/info/refs [/quote]
You can’t clone direct links to files. To get the link for cloning got to the repository’s main page, i.e. go to:

Search for the text “HTTPS clone URL”. It will show:

To clone:

git clone https://github.com/Whonix/msgcollector.git

I’ll be testing your code.

Anyhow, if you make the commit, you get the credit. Otherwise I falsely get it. I don’t mind to commit the code you posted here, but I think it’s better if you do it.

11
Anyhow, if you make the commit, you get the credit. Otherwise I falsely get it. I don't mind to commit the code you posted here, but I think it's better if you do it.

You can commit, no harm, there will be more to come when I am comfortable with git. The popup does not show in error_handler(), though, so you should remove the line calling ‘dispatch_X.py’. The zenity replacement in dispatch_x_active has been thoroughly checked.

12

It works well.

What do you think about the move to left top feature? whonixcheck progress bar and result window generally is in left top. Other windows, such as from timesync are normal (centric). This is especially useful so they don’t overlap.

Could you move the window to the left top using python?

When,

[code]if [ -f “/var/run/msgcollector/${identifier}.lefttop” ]; then

fi[/code]

is true, you could just add an extra argument. Such as,

if [ -f "/var/run/msgcollector/${identifier}.lefttop" ]; then local maybe_lefttop maybe_lefttop="--lefttop" fi

/usr/lib/whonix/dispatch_X.py --type "$type" --title "$title" --msg "$msg" "$maybe_lefttop"

What about the favicon? (The small icon in the left top?) --icon “$icon”? Where $icon contains the path to the icon?

13

Pushed the new msgdipatcher at GitHub - troubadoour/msgcollector.

Have added the $lefttop feature, but for reasons difficult to explain here (it’s difficult to follow Qt’s logic when it comes to moving windows around the screen), it has become bottom right when timesync and whonixcheck popups may hide each other. It should not be a big issue if we rename ‘/var/run/msgcollector/${identifier}.lefttop’ to ‘/var/run/msgcollector/${identifier}.bottomright’ in the scripts.

For the moment, I did not manage to display the flavicon in the popup title bar, may be a size problem, and the script does not run when it’s called from error_handler(). Still a mystery, but since ii popped up a couple of times, it might be working for you.

14

What python debian packages must be installed, i.e. what are the dependencies for your script?

What do you think about a no-filenames-with-uppercase-chars policy?

Please use an editor that shows or even automatically deletes leading spaces.

I merged your changes and pushed a few on top of that. Please get them.

(Paths are changed to match the newly spitted packages, because msgcollector will be a fully standalone package. Making good progress. This breaks compatibly with for example whonixcheck in Whonix 8, but will work again in Whonix 9, because then whonixcheck paths are updates as well. Due to split packages, installing Whonix could become almost as simple as “sudo apt-get install whonix-workstation”.)

bottomright vs lefttop is a bit sad. Do you think you can fix this and the favicon? Keep your time, maybe inspiration will come later, will be still plenty of time before Whonix 9.

Still a mystery, but since ii popped up a couple of times, it might be working for you.
Most likely fixed. (https://github.com/Whonix/msgcollector/commit/d88d8dcc6467f06beb8779ed1026250889613dc4) (Sometimes it's best to simulate the command by copy and pasting a likely example [variables] in the shell. Sometimes I get some "live" variables first before trying.)
15
What python debian packages must be installed, i.e. what are the dependencies for your script?
Pushed troubadoour/msgcollector with python-qt4 in debian/control Depends.
What do you think about a no-filenames-with-uppercase-chars policy?
Do you mean something like "msgCollector", "msgCollectorDispatchX", or timesyncPostError?
Please use an editor that shows or even automatically deletes leading spaces.
I'm using kate for everything from now on. I don't know if it can do that. Can you tell where I was wrong?
bottomright vs lefttop is a bit sad. Do you think you can fix this and the favicon? Keep your time, maybe inspiration will come later, will be still plenty of time before Whonix 9.
I'll work on that, it is definitely feasible. Ant idea when Whonix 9 should be released?
Most likely fixed. (https://github.com/Whonix/msgcollector/commit/d88d8dcc6467f06beb8779ed1026250889613dc4)
Yeah... fixed and tested.
16

Added a minor commit on top of yours. (No more than 80 lines per debian/control - I thought that’s a convention for debian/control files, but looks like I was wrong. Nevermind)

Do you mean something like "msgCollector", "msgCollectorDispatchX", or timesyncPostError?
I am suggesting to use msgdispatcher_dispatch_x (all lower case) instead of msgdispatcher_dispatch_X (X is upper case). All other file names in Whonix's code are also lower case only. I guess when always using lower case, there is less chance of messing it up somewhere. As for file names, I most times use copy and paste, so this is also only a minor point.
I'm using kate for everything from now on. I don't know if it can do that. Can you tell where I was wrong?
Example: https://github.com/Whonix/msgcollector/commit/28d6a61abd512611c7b150bd89e5862d767b6456

With Kate you’re lucky. It can show trailing spaces as well as automagically remove them when saving.

Kate Setting (version 4.8.4-1):

  • Kate → Settings → configure Kate → check “highlight tabulators” and “highlight trailing spaces” (all recent
  • Kate → Settings → open/save → check “Remove trailing spaces”
Ant idea when Whonix 9 should be released?
Not sure. After splitting packages with new msgcollector would be worth a new release. Maybe in two or three months.
17

Pushed msgcollector with “msgdispatcher_dispatch_x”.

It seems that the window position problem was caused by Xfce, or my combination of Xfce and KDE, I’ll have to check. Anyhow, I have reverted to KDE for GUI programming and the logic is back. When $lefttop is set, It moves to the leftop position now. The proper flavicon is displayed too.

Now, as the result messages from timesync and whonixchek are handled by the same script, they cannot be displayed at the same time. If they are running at the same time, the user has to close one for the other one to show, hence they will never overlap. We could put back both at the center of the screen, it would be more consistent and would look more urgent.

18

I also pushed some changes as well, please get them.

We could put back both at the center of the screen, it would be more consistent and would look more urgent.
whonixcheck/timesync both could show results after boot. Having them overlap was confusing in past.

Doing a testrun (from msgcollector folder):

usr/lib/msgcollector/msgdispatcher_dispatch_x "warning" "test" "test msg" "1" "/usr/share/whonix/icons/whonix.ico"

Please move the window elsewhere. Below that window, there is another empty useless window. Can you remove this?

they cannot be displayed at the same time
Blocking all output of msgdispatcher (X + CLI) as long as the user doesn't click a window sounds like a major bug. It's supposed to become a general message notification dev toolkit.

To show multiple windows at the same time, we could simply launch msgdispatcher_dispatch_x into the background. Example:

usr/lib/msgcollector/msgdispatcher_dispatch_x "warning" "test" "test msg" "1" "/usr/share/whonix/icons/whonix.ico" &

(We just need to add a “&” behind our current invocations of msgdispatcher_dispatch_x.)

I could even add some job control to check msgdispatcher_dispatch_x’s exit code after a few seconds to log any eventual failures.

19
Please move the window elsewhere.
Where is it showing? Not at the left top?
20

Where is it showing? Not at the left top?[/quote]
It is. But that was not what I was talking about.

When you run dispatch_x once, you already get two popups. The real one on top. Please move it elsewhere. You. With your mice. Just to see it. Not with programming. Just to look what I mean. Then you’ll see that there is a second popup below the real popup. The second one should get rid of. I can make a screenshot if necessary.