ActiveState Community Site - Stackless Python support - Comments

Stackless/dbgp hang/crash

DarrinWest — Mon, 11 Feb 2008 12:05:00 -0800

I built a debug version of stackless from here: http://www.stackless.com/binaries/stackless-251-export.tar.bz2

And I've been applying the suggested changes from this forum thread, but still have not gotten success.

The break within the tasklet *does* work, but then hangs before printing "after".

If I remove the brk, the tasklet completes, and stackless.run() returns, to print "after", as I would expect.

import sys
print sys.path
import dbgp.client
import stackless

print "start"

def f():
    dbgp.client.brk(host="localhost", port=9666) # DRW: this works
    print "hello"

task = stackless.tasklet(f)() # note the extra () cause it to "bind"
foo = stackless.run()
print "after"

$ ./python.exe
Python 2.5.1 Stackless 3.1b3 060516 (release25-maint, Feb  6 2008, 11:27:17) [MS
C v.1400 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print sys.path
['', 'c:\\sandbox\\SrvML\\Framework\\Developer\\External\\Code\\Python\\Windows\
\python25.zip', 'C:\\sandbox\\srvml\\Framework\\Developer\\External\\Code\\Pytho
n\\Windows\\DLLs', 'C:\\sandbox\\srvml\\Framework\\Developer\\External\\Code\\Py
thon\\Windows\\lib', 'C:\\sandbox\\srvml\\Framework\\Developer\\External\\Code\\
Python\\Windows\\lib\\plat-win', 'C:\\sandbox\\srvml\\Framework\\Developer\\Exte
rnal\\Code\\Python\\Windows\\lib\\lib-tk', 'c:\\sandbox\\SrvML\\Framework\\Devel
oper\\External\\Code\\Python\\Windows', 'C:\\sandbox\\srvml\\Framework\\Develope
r\\External\\Code\\Python\\Windows\\lib\\site-packages']
>>> import dbgp.client
Darrin: non-compiled client.py
>>> import stackless
>>>
>>> print "start"
start
>>>
>>> def f():
...     dbgp.client.brk(host="localhost", port=9666) # DRW: this works
...     print "hello"
...
>>> task = stackless.tasklet(f)() # note the extra () cause it to "bind"
>>> foo = stackless.run()
hello

When I embed the VM in a C++ app (visual studio 2005, on XP), I get a crash:

in
hello
Assertion failed: Py_UnwindToken->tempval == NULL, file ..\Stackless\module\sche
duling.c, line 460

 	msvcr80d.dll!_wassert(const wchar_t * expr=0x1e23d218, const wchar_t * filename=0x1e23d1d0, unsigned int lineno=460)  Line 212	C
 	python25_d.dll!restore_tracing(_frame * f=0x00000000, int exc=0, _object * retval=0x00000000)  Line 460 + 0x21 bytes	C
 	python25_d.dll!slp_frame_dispatch_top(_object * retval=0x00000000)  Line 689 + 0x12 bytes	C
 	python25_d.dll!slp_run_tasklet()  Line 1155 + 0x9 bytes	C
 	python25_d.dll!slp_eval_frame(_frame * f=0x040bbd88)  Line 299 + 0x5 bytes	C
 	python25_d.dll!climb_stack_and_eval_frame(_frame * f=0x040bbd88)  Line 266 + 0x9 bytes	C
 	python25_d.dll!slp_eval_frame(_frame * f=0x040bbd88)  Line 294 + 0x9 bytes	C
 	python25_d.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x03f5f568, _object * globals=0x040bbd88, _object * locals=0x00000000, _object * * args=0x03dd1afc, int argcount=4, _object * * kws=0x00000000, int kwcount=0, _object * * defs=0x00000000, int defcount=0, _object * closure=0x00000000)  Line 3125 + 0x6 bytes	C
 	python25_d.dll!function_call(_object * func=0x03f56e50, _object * arg=0x03dd1ae8, _object * kw=0x00000000)  Line 525 + 0x40 bytes	C
 	python25_d.dll!PyObject_Call(_object * func=0x03f56e50, _object * arg=0x03dd1ae8, _object * kw=0x00000000)  Line 1863 + 0x3c bytes	C
 	python25_d.dll!PyEval_CallObjectWithKeywords(_object * func=0x03f56e50, _object * arg=0x03dd1ae8, _object * kw=0x00000000)  Line 3745 + 0x11 bytes	C
 	python25_d.dll!PyObject_CallObject(_object * o=0x03f56e50, _object * a=0x03dd1ae8)  Line 1852 + 0xf bytes	C
 	Entity.dll!egf::Entity::DoScriptBehavior(eut::SmartPointer spPendBehavior={...}, eut::IDataStream * pArguments=0x0185642c)  Line 1971 + 0x11 bytes	C++

+		f	0x00000000 {_ob_next=??? _ob_prev=??? ob_refcnt=??? ...}	_frame *
+		retval	0x00000000 {_ob_next=??? _ob_prev=??? ob_refcnt=??? ...}	_object *
-		ts	0x03d46d90 {next=0x00000000 interp=0x03d46ce8 frame=0x00000000 ...}	_ts *
+		next	0x00000000 {next=??? interp=??? frame=??? ...}	_ts *
+		interp	0x03d46ce8 {next=0x00000000 tstate_head=0x040e2a40 modules=0x03da7d58 ...}	_is *
+		frame	0x00000000 {_ob_next=??? _ob_prev=??? ob_refcnt=??? ...}	_frame *
		recursion_depth	0	int
		tracing	0	int
		use_tracing	1	int
		c_profilefunc	0x00000000	int (_object *, _frame *, int, _object *)*
		c_tracefunc	0x1e1576c0 trace_trampoline(_object *, _frame *, int, _object *)	int (_object *, _frame *, int, _object *)*
+		c_profileobj	0x00000000 {_ob_next=??? _ob_prev=??? ob_refcnt=??? ...}	_object *
+		c_traceobj	0x03ebb2f8 {_ob_next=0x03f5e038 _ob_prev=0x03f82df0 ob_refcnt=1 ...}	_object *
+		curexc_type	0x1e288a38 _PyExc_TaskletExit {_ob_next=0x03dbab78 _ob_prev=0x1e2887e0 ob_refcnt=9 ...}	_object *
+		curexc_value	0x03da5038 {_ob_next=0x1e2d9148 _ob_prev=0x03da6028 ob_refcnt=2489 ...}	_object *
+		curexc_traceback	0x00000000 {_ob_next=??? _ob_prev=??? ob_refcnt=??? ...}	_object *
+		exc_type	0x00000000 {_ob_next=??? _ob_prev=??? ob_refcnt=??? ...}	_object *
+		exc_value	0x00000000 {_ob_next=??? _ob_prev=??? ob_refcnt=??? ...}	_object *
+		exc_traceback	0x00000000 {_ob_next=??? _ob_prev=??? ob_refcnt=??? ...}	_object *
+		dict	0x03ebf4d0 {_ob_next=0x03ec4418 _ob_prev=0x03eb8178 ob_refcnt=1 ...}	_object *
		tick_counter	663	int
		gilstate_counter	1	int
+		async_exc	0x00000000 {_ob_next=??? _ob_prev=??? ob_refcnt=??? ...}	_object *
		thread_id	4748	long
+		st	{initial_stub=0x03dd9028 serial=50 serial_last_jump=50 ...}	_sts
+		ts->frame	0x00000000 {_ob_next=??? _ob_prev=??? ob_refcnt=??? ...}	_frame *


-		Py_UnwindToken	0x1e243fe4 unwind_token {_ob_next=0x00000000 _ob_prev=0x00000000 ob_refcnt=1 ...}	PyUnwindObject *
+		_ob_next	0x00000000 {_ob_next=??? _ob_prev=??? ob_refcnt=??? ...}	_object *
+		_ob_prev	0x00000000 {_ob_next=??? _ob_prev=??? ob_refcnt=??? ...}	_object *
		ob_refcnt	1	int
+		ob_type	0x1e243ff8 PyUnwindToken_Type {_ob_next=0x00000000 _ob_prev=0x00000000 ob_refcnt=1 ...}	_typeobject *
-		tempval	0x1e2b38ac __Py_NoneStruct {_ob_next=0x03db6ca0 _ob_prev=0x03db6cd8 ob_refcnt=2260 ...}	_object *
+		_ob_next	0x03db6ca0 {_ob_next=0x03dbd028 _ob_prev=0x1e2b38ac ob_refcnt=247 ...}	_object *
+		_ob_prev	0x03db6cd8 {_ob_next=0x1e2b38ac _ob_prev=0x1e2c75d4 ob_refcnt=2 ...}	_object *
		ob_refcnt	2260	int
+		ob_type	0x1e2b36a8 PyNone_Type {_ob_next=0x03db37d0 _ob_prev=0x03db61f8 ob_refcnt=6 ...}	_typeobject *

I am going to apply the "traceDispatch" suggestion above, but my stuff doesn't seem to have the same indications. I was hoping someone could glance at this and see what is up.

Tracing tasklets.

vilhelm — Wed, 24 Oct 2007 12:08:58 -0700

The tracing for tasklets needs to be set for every tasklet that is run.

So you would have to override the __call__ method for the stackless tasklets to enable tracing before it is executed.

There is also the problem when tasklets have been running before you start tracing, then you have to call set_trace() within that tasklet and update the frame.f_trace of the execution frame to point to your trace function to enable tracing.
This can be done by using the stackless.set_schedule_callback to listen in on all context switches and setting the trace before it runs again.

import sys
import stackless

def contextDispatch( prev, next ):
    if not prev: #Creating next
        pass
    elif not next: #Destroying prev
        pass
    else:
        # Prev is being suspened
        # Next is resuming
        if not next.frame.f_trace:
            # We might allready be tracing so ...
            sys.call_tracing(next.settrace,(traceDispatch,))


stackless.set_schedule_callback( contextDispatch  )

def __call__(self, *args, **kwargs):
     f = self.tempval
     def new_f(old_f, args, kwargs):
         sys.settrace(traceDispatch)
         old_f(*args, **kwargs)
         sys.settrace(None)
     self.tempval = new_f
     stackless.tasklet.setup(self, f, args, kwargs)

def settrace( self, tb ):
    self.frame.f_trace = tb   
    sys.settrace( tb )

stackless.tasklet.__call__ = __call__
stackless.tasklet.settrace = settrace

The 'traceDispatch' should be the trace function of the python debugger you are using.

Vilhelm

Yes, breakpoints in a tasklet were also ignored on Linux

ToddW — Mon, 24 Sep 2007 10:38:40 -0700

I tested this using the ping_pong sample code from here:
http://members.verizon.net/olsongt/stackless/why_stackless.html#pingpong...

Same problem exists on the Linux machine as to the Windows machine, the breakpoints inside of the "ping" and "pong" functions were ignored.

Todd.

Tasklet problem

adkilla — Sun, 23 Sep 2007 19:33:38 -0700

I also noted that breakpoints that are set in a stackless tasklet were ignored, I'm not sure why (something to do with the (micro-)threading implementation I would guess).

Do you have this problem on Linux as well? I've also noticed that psyco will crash if used in conjunction with stackless.

-Ad

Your right

ToddW — Thu, 20 Sep 2007 12:58:19 -0700

On Windows the debugger crashed (I had only tested the debugging on Linux). There is a workaround for this though, and that is to force the use of the Python version of the client debugger (by default the client debugger uses a C-extension module). I changed my dbgp "client.py" file, located at:

C:\Program Files\ActiveState Komodo IDE 4.2\lib\support\dbgp\pythonlib\dbgp

to look like this:

--- client_orig.py Thu Sep 20 12:51:15 2007

+++ client.py Thu Sep 20 12:51:26 2007

@@ -55,7 +55,7 @@

 # Import the client support module (implemented in C for some Python

 # versions for speed).

-if sys.hexversion < 0x02020000:

+if sys.hexversion < 0x02020000 or sys.version.split(None, 2)[1:2] == ["Stackless"]:

     # We don't compile _clientXY for pre-2.2 Python.

     from dbgp._pyclient import *

 else:

--- client_orig.py Thu Sep 20 12:51:15 2007
+++ client.py Thu Sep 20 12:51:26 2007
@@ -55,7 +55,7 @@
 
 # Import the client support module (implemented in C for some Python
 # versions for speed).
-if sys.hexversion < 0x02020000:
+if sys.hexversion < 0x02020000 or sys.version.split(None, 2)[1:2] == ["Stackless"]:
     # We don't compile _clientXY for pre-2.2 Python.
     from dbgp._pyclient import *
 else:

This got around the crash problem.

I also noted that breakpoints that are set in a stackless tasklet were ignored, I'm not sure why (something to do with the (micro-)threading implementation I would guess).

How about debugging?

adkilla — Wed, 19 Sep 2007 19:30:14 -0700

Could you please try loading a python script, setting some breakpoints and stepping through them using the Komodo debugger? Does it crash Stackless Python?

I wiped a spare PC clean and re-installed XP, official Python 2.5.1 with Stackless Python 2.5.1 and Komodo 4.2 from scratch. I've noticed that if I load a source file with a breakpoint, the interactive shell and debugger will crash. If I don't set any breakpoints and just run the python source file, it will work fine. I seriously doubt that my setup is a problem this time.

Thanks very much for your kind feedback, you've been most helpful.

-Ad

Re: Python version

ToddW — Tue, 18 Sep 2007 21:57:12 -0700

I used the python 2.5.1 msi installer from python.org.

Which Python distro?

adkilla — Tue, 18 Sep 2007 18:37:04 -0700

I unzipped stackless over ActivePython 2.5.1.1. Which python distribution are you using?

Yes, mine works fine from the command prompt.

Thanks,
-Ad

Just tested Windows

ToddW — Tue, 18 Sep 2007 10:59:39 -0700

Interesting, on windows XP, I just installed stackless (unzipped stackless over a Python 2.5.1 install) and it worked perfectly with Komodo IDE 4.2 beta7. Ran the tutorial code examples without a problem.

Python 2.5.1 Stackless 3.1b3 060516 (release25-maint:55039:55048, May  1 2007, 14:13:4) [MSC v.1310 32 bit (Intel)] on win32

Does your stackless work from the command prompt?

Built from source

ToddW — Tue, 18 Sep 2007 09:09:42 -0700

Yes, I downloaded the tarball and built it from source on my Linux machine.

Stackless Info

adkilla — Mon, 17 Sep 2007 23:32:27 -0700

I am using Windows XP. Here is my Stackless Python build info:

Python 2.5.1 Stackless 3.1b3 060516 (release25-maint:55039:55048, May  1 2007, 14:13:4) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

Did you compile it from scratch?

Thanks,
-Ad

Curious

ToddW — Thu, 13 Sep 2007 11:14:46 -0700

Which platform are you using? I'm using Linux (Ubuntu) and the Python debugging and interactive shell work with my stackless build:

Python 2.5.1 Stackless 3.1b3 060516 (release25-maint, Sep 11 2007, 10:18:57)

Todd.

Some pointers needed

adkilla — Thu, 13 Sep 2007 02:53:34 -0700

Thanks for the reply. I could look into adding the support but I would need some pointers, like where to start looking. I tried debugging the debugger in support\dbgp\pydbgp.py in Komodo but it didn't work. Some info on how to debug the debugger would help a lot as well.

Thanks,
-Ad

Komodo only supports standard Python

shanec — Tue, 11 Sep 2007 07:16:30 -0700

At this time we only support standard python. The debugger interfaces and behaviors are different for Stackless Python or IronPython.

The DBGP implementation for python that is used in Komodo is open source, under MIT license, if you wish to try and make it work, we'd be happy to answer any questions you might have.

Regards,
Shane Caraveo.

Stackless Python support

adkilla — Mon, 10 Sep 2007 19:17:52 -0700

Hi,

I need Stackless Python 2.5.x support in Komodo. I've tried using the recent Komodo 4.2 beta7 release without success. The interactive shell and debugger crashes Stackless Python on start.

Any plans to fix this?

Thanks,
-Ad