ActiveState Community Site - The Impatient Developer's Guide to Writing Python Nose Plugins - Comments

Ugh... there's something

yaneurabeya — Mon, 03 Nov 2008 22:23:41 +0000

Ugh... there's something fishy going on here too with the above example and our plugin... investigating further.

Coding suggestion for formatErr with v0.10.3/0.10.4 incorrect

yaneurabeya — Mon, 03 Nov 2008 03:21:10 +0000

Using the formatErr suggested above, you'll run into weird issues with package setup and teardown if you insert an error in __init__.py, etc, like I describe in:

http://code.google.com/p/python-nose/issues/detail?id=217

Following logic similar to the following should work better (this is from 0.10.3's nose.plugins.capture; FYI -- this is LGPL'ed text):

def formatError(self, test, err):
        test.capturedOutput = output = self.buffer
        self._buf = None
        if not output:
            # Don't return None as that will prevent other
            # formatters from formatting and remove earlier formatters
            # formats, instead return the err we got
            return err
        ec, ev, tb = err
        return (ec, self.addCaptureToErr(ev, output), tb)

I'll post my attempt on the list too...

ericp — Tue, 29 Apr 2008 18:09:20 +0100

... unless there's an outpouring of interest from Komodo community
members on writing Nose plug-ins, it makes total sense to move the
discussion there.

For those interested, see http://groups.google.com/gropu/nose-users?hl=en

Thanks Kumar, I will post

bcorfman — Tue, 29 Apr 2008 15:01:48 +0100

Thanks Kumar, I will post there once I can refer to the code.

There might be a few more

kumar303 — Tue, 29 Apr 2008 13:51:46 +0100

There might be a few more eyes on it if you post to the nose-users list

Still in progress

bcorfman — Tue, 29 Apr 2008 13:34:05 +0100

I created a new 1.2 branch for kNose so I could continue to experiment with using the nose API while still maintaining the 1.1.x version. I'm at work right now and don't have access to the 1.2 code, but if you provide me a place to send it or upload it, I'll get you my code tomorrow.

I have been completely successful with calling nose.run() from a Python shell, but only partially successful when calling it inside kNose. Inside the extension, nose will correctly return pass/fail/error on any test classes derived from unittest.TestCase, but it will always report success on standalone test functions, i.e.

def testMe():
    assert (1==0) # this should fail, but it doesn't

I figure it has something to do with how the environmental variables are set within nose, but right now I'm at a loss, even when stepping through in the Komodo debugger. Some nose options cancel out others, and the documentation isn't much help in this regard. I have reported the behavior on the nose issues list (http://code.google.com/p/python-nose/issues/list), but I haven't gotten a response yet.

I got a recommendation from Kumar to call nose.main() using subprocess, instead of nose.run(). That's not much different than what I'm doing in knose 1.1 though, and it's adding a lot of complexity just to call plugins. I'd really rather get nose.run() working directly if there's any way to do so since that actually simplifies my code.

That's the update. Let me know if you want to see what I have so far. I may be missing something obvious, and if you can suggest a fix, that would be great.

Is there more to running nose plugins without installing them?

ericp — Mon, 28 Apr 2008 22:59:43 +0100

Hi Brandon,

I've revisited the Nose integration, working on making the tests extensible.
They work fine using the setup script, but I can't get them to work using
your technique a couple of comments back. Is there more to it? It seems
that using that technique, Nose won't load any tests, even when I pass
them as an explicit argv= in the call to nose.run

Thanks,
Eric

I was almost finished with

bcorfman — Mon, 31 Mar 2008 20:22:32 +0100

I was almost finished with updating kNose to use the plugin API, but I ran into a snag. It turns out that nose imports all unit test modules into memory and keeps them there if you use a nose.run() call. As a result, if you call nose.run() a second time, you will get stale information because the unit tests will not get reloaded into memory.

You must use a '--with-isolation' flag in order to enable nose's IsolationPlugin; this makes nose unload the unit test modules after it completes each test. Unfortunately, this seems to work only with test classes derived from unittest.TestCase. If you have basic functions inside a module instead, these will still not get reloaded correctly.

I've got a couple of requests for help out to the TIP list and the nose issue tracker on Google Code. I will post an update here if I found out anything.

Eric, I've been

bcorfman — Thu, 20 Mar 2008 13:47:15 +0000

Eric, I've been investigating your techniques in order to update kNose. You probably have found this out already, but your article here is referencing the nose 0.9 API, not the new 0.10 API. The new version of nose changes the plugin API (and the method signatures for your KomodoOutput class above).

However, a very exciting thing about nose 0.10 is that it no longer requires plugins to be installed with a setup script. You can simply pass the plugin class to the nose.run command with a 'plugins' keyword argument, like so:

import nose
class KomodoOutput(Plugin):
   ...

nose.run(plugins=[KomodoOutput()])

The new API is much more plugin friendly as a result.

going further...

kumar303 — Tue, 26 Feb 2008 22:52:30 +0000

Cool. Excellent quickstart guide. If you want to get the plugin working for nose >= 0.10 or write some tests for the plugin (always a good idea!) then there are instructions and examples on this page of the nose docs: http://somethingaboutorange.com/mrl/projects/nose/doc/writing_plugins.ht...

The Impatient Developer's Guide to Writing Python Nose Plugins

ericp — Sat, 23 Feb 2008 01:38:58 +0000

In between fixing bugs for the impending 4.3 release, I found some time over the last couple of days to figure out how to build a plugin for Nose, a Python test framework that sits on top of unittest.py. This entry will talk about the why and the how.

First, we've added a unit-test framework to Komodo. The Python framework currently handles only tests based on unittest.py (aka PyUnit). You have to tell it which tests to load, which is a bit awkward. Nose has the smarts to find testable entities in a directory, so we don't need to duplicate that.

Also, Nose supports plugins. These plugins fall into two camps: plugins that find new things to test, and plugins that filter the output from the test runs.

Some of you might know that Brandon Corfman wrote a Komodo extension for Nose, available at http://community.activestate.com/xpi/knose. It's a good extension, but unfortunately doesn't work with the new Komodo framework. Brandon also wrote it by parsing the raw output from Nose. As I've learned from adding Perl testing to Komodo, this is a hard path to follow. It's always preferable to override an existing class, where you get to decide which output should be emitted for a set of events. Whenever possible, choose to emit output over parsing input.

The problem is that the documentation on writing Nose plugins is a bit terse, and makes a few assumptions on what the reader knows. Here's what I worked out, the "how" of our story. It's actually surprisingly short.

You need to create two files (at least): your plugin, and a setup.py file. I won't give all the details of the plugin (you can see the full code once it ships with Komodo), but I'll give the essentials here.

First, you need some boilerplate to override the nose.plugins.Plugin class. I defined the following class in a file called komodoplug.py:

importtraceback
import nose
from nose.pluginsimport Plugin
class KomodoOutput(Plugin):

    """

    Output test results in a format suitable for Komodo's unit-testing

    framework.

    """

    # Required attributes:

    name = 'komodo'   # invoke with --with-komodo argument to Nose

    enabled = True

import traceback
import nose
from nose.plugins import Plugin

class KomodoOutput(Plugin):
    """
    Output test results in a format suitable for Komodo's unit-testing
    framework.
    """
    # Required attributes:
    
    name = 'komodo'   # invoke with --with-komodo argument to Nose
    enabled = True

In the same class, I then redefined the methods I wanted to provide output for:

def addSuccess(self, test, capt):
self.stream.writeln("passed")
    def addError(self, test, err, capt):

        self.stream.write("error: ")

        self._writeFault(err)

    def addFailure(self, test, err, capt, tb_info):

        self.stream.write("failure: ")

        self._writeFault(err")
    def _writeFault(self, err):

        self.stream.writeln("".join(self.formatErr(err)))
    def formatErr(self, err):

        exctype, value, tb = err

        return traceback.format_exception(exctype, value, tb)

    def setOutputStream(self, stream):

        # grab for own use

        self.stream = stream

        # return dummy stream to suppress default output

        class dummy:

            def write(self, *arg):

                pass

            def writeln(self, *arg):

                pass

        d = dummy()

        return d

    def startTest(self, test):

        # There are some "start" things Nose tells us about

        # that we don't want to hear about:

        if isinstance(test, (nose.core.TestCollector,

                             nose.suite.TestClass,

                             nose.suite.TestDir,

                             nose.suite.TestModule)):

            return

        self.stream.write("test %s..." % test)

def addSuccess(self, test, capt):
        self.stream.writeln("passed")

    def addError(self, test, err, capt):
        self.stream.write("error: ")
        self._writeFault(err)
            
    def addFailure(self, test, err, capt, tb_info):
        self.stream.write("failure: ")
        self._writeFault(err")

    def _writeFault(self, err):
        self.stream.writeln("".join(self.formatErr(err)))

    def formatErr(self, err):
        exctype, value, tb = err
        return traceback.format_exception(exctype, value, tb)
    
    def setOutputStream(self, stream):
        # grab for own use
        self.stream = stream
        # return dummy stream to suppress default output
        class dummy:
            def write(self, *arg):
                pass
            def writeln(self, *arg):
                pass
        d = dummy()
        return d
    
    def startTest(self, test):
        # There are some "start" things Nose tells us about
        # that we don't want to hear about:
        if isinstance(test, (nose.core.TestCollector,
                             nose.suite.TestClass,
                             nose.suite.TestDir,
                             nose.suite.TestModule)):
            return
        self.stream.write("test %s..." % test)

The part that was new for me is that plugins need to be installed for Nose to find them. Here's the setup.py file, located in the same directory as komodoplug.py.

importsys
try:
import ez_setup

    ez_setup.use_setuptools()
exceptImportError:
pass
from setuptools import setup
setup(

    name='Komodo output plugin',

    version='0.1',

    author='Eric Promislow',

    author_email = 'ericp@activestate.com',

    description = 'Komodo output',

    license = 'ActiveState Komodo',

    py_modules = ['komodoplug'],

    entry_points = {

        'nose.plugins': [

            'komodoplug = komodoplug:KomodoOutput'

            ]

        }
    )

import sys
try:
    import ez_setup
    ez_setup.use_setuptools()
except ImportError:
    pass

from setuptools import setup

setup(
    name='Komodo output plugin',
    version='0.1',
    author='Eric Promislow',
    author_email = 'ericp@activestate.com',
    description = 'Komodo output',
    license = 'ActiveState Komodo',
    py_modules = ['komodoplug'],
    entry_points = {
        'nose.plugins': [
            'komodoplug = komodoplug:KomodoOutput'
            ]
        }

    )

I then ran easy_setup . in the same directory as these two files, and then was ready to try it out. I switched to the directory I downloaded BeautifulSoup into, and ran the following command:

nosetests --with-komodo  BeautifulSoupTests.py

and got my own test-result output.

Let's try it with one of the Komodo source files:

C:>nosetests --with-komodo reflow.py
Done: run=0 errors=0 failures=0; T: 0.078 seconds

This isn't the output you'll get from the code snippet above, but it should give the idea that nothing happened. That's because this module uses doctest for testing, so we need to tell Nose where to find the tests. If I try this command-line:

C:>nosetests --with-komodo --with-doctest reflow.py

I get the results of the doc-based tests in the output format my plugin generated. This works because the Komodo plugin is an output-based plugin, while the doctest is a plugin that determines which tests to load. The two plugins combine seamlessly. Very cool.