[HOME] [DOWNLOAD] [DOCS] [NEWS] [SUPPORT] [TIPS] [ISSUES] [DONATE]

automated test suite

Not written in stone, means, we can discuss this further.

Tails has an automated test suite:
https://tails.boum.org/contribute/release_process/test/automated_tests/

Whonix needs an automated test suite as well.

  1. Check, if https://github.com/Whonix/Whonix/blob/master/CONTRIBUTING.md is acceptable.
  2. Create an automated test suite and/or adapt whatever is useful from the Tails’ automated test suite.
  3. Share it on git(hub).
  4. Optional: package as .deb.

9 posts were split to a new topic: Qubes-Whonix using openQA Instance by Qubes

I am starting work on this

2 Likes

A few questions

  1. How would you like this repository set up? I have whonix forked, and could add automated testing in a seperate folder. I can create a public repository as well and am happy to give write access to @Patrick

  2. Do we want a jenkins server set up as a CI/CD pipe for source code PRs? Jenkins is a bit new to me but I am happy to take a stab at this.

  3. I would prefer to write ruby, and follow their model of

--Given /^I capture all network traffic$/ do
    @sniffer = Sniffer.new("sniffer", $vmnet)
    @sniffer.capture
    add_after_scenario_hook do
      @sniffer.stop
      @sniffer.clear
    end
  end

but without all the ivars

Once it is configured, I feel confident I can write effective tests productively. But getting it spun up may be a bit of a challenge

1 Like

This overlaps with this recent iniative:

Seems active. Last commit less than 24 hours ago.

Could you please join that forum thread and view that git repository for your oversight?

That is, because I am not the most senior python programmer and also this test suite seems uncharted territory (at least for me). Therefore it would be good if we can give good feedback to the students early so fundamental issues are noticed early.

2 Likes

I will join the thread and do my best. I was really hoping to do this in ruby, as I am very experienced with ruby (many thousands of hours). Ive only spent a few dozen hours or so with python. But the same principles should apply I think.

I will start the conversation with them.

1 Like

Automated Test Suite Updates:

Due to lack of communication from the college crew, I started throwing together a suite with @Patrick’s blessing.

I currently have CI setup and basic build checks passing.

Everyone out there on the world wide is welcome to follow along, contribute, critique, etc.

Next Steps:

  • get the VMs working properly (libvirt)
  • point and click functionality to perform true BDD (dogtail & sikuli)
  • create automated project board with test cases
  • take a stab at kernel testing while likely annoying patrick with lots of noob questions around the kernel (hooray for learning!)

Here’s the repo:

And the CI builds:
https://travis-ci.com/Mycobee/whonix_automated_test_suite/builds/

1 Like

Awesome!

Yes, we can have two automated test suites. Either fully independent or perhaps later integrated somehow (probably not important). The work on the other test suite shouldn’t block any progress here.

These questions would be ideally asked in this thread kernel recompilation for better hardening or if not suitable there in a separate thread that is because @madaidan, not me did most of that work.

1 Like

Project Updates:

Overview:

After discussion with @Patrick and @fahr3n, it seems as though we have a more clear idea of how to move forward and what we can do to actually make this happen. At a high level my primary objectives are:

  1. Successfully build Whonix VM with Vagrant in CI
  2. Configure CI to run @fahr3n’s feature tests
  3. Start writing Kernel Tests

Current Progress, Plans, and Challenges:

After successfully configuring vagrant and libvirt to work in my environment, I began researching heavily what it will take to run an automated testing suite with CI using a Vagrant VM. Essentially, I have having to pivot the CI tooling from TravisCI to Jenkins for this job.

Jenkins already has a plugin for Vagrant for this sort of thing, and it seems nearly impossible with TravisCI.

With that said, these are the action items required from me to move forward with objective #1.

  • Stand up local jenkins server (done)
  • Connect vagrant plugin with Jenkins to build VM in CI environment (in progress)
  • Successfully run automated tests in CI VM

After that is complete, I plan to start integrating test cases.

Feedback, comments, concerns, and questions are all welcome

1 Like
[Imprint] [Privacy Policy] [Cookie Policy] [Terms of Use] [E-Sign Consent] [DMCA] [Investors] [Priority Support] [Professional Support]