This is a hard problem (as evidenced by Greg Stein and Mark Hammond?’s efforts in 1999) mainly from a performance point of view. GvR himself has stated that although he doesn’t have the time to do it himself, he would be willing to accept any patches to CPython which removes the GIL (as a compile time option) which does not negatively affect performance of single threaded applications and significantly improves I/O bound multi-threaded apps.

As I learnt at PyCon 2007, the problems in general have to do with Python’s garbage collection mechanism which uses reference counting and hence requires synchronized access to objects to ensure that the increfs and decrefs are done properly. Additionally, the GIL protects various caches (notably the integer (and some other types?) free lists so that creating and destroying those items don’t take too much time because the PyObjects are recycled.

So far, the solutions discussed involve some type of fine grained locking around critical code sections. The problem here is that beyond two or three cores, this doesn’t scale well because of lock contention (since so much stuff needs to be synchronized).

Well, what if we tried a different approach… Locks don’t seem like they’re going to solve the problem in a scalable way and if we’re going to take the effort to do this, might as well do it so that the benefits scale in an environment of more than 100 cores (its possible – see Azul Systems).

What if we dedicated one core for running synchronized code? Naturally this can’t be done without support from the kernel (perhaps some sort of kernel module/VxD).

At process initialization time, the kernel dedicates a single core to be the “synchronization server” and provides an instruction queue for it. This can vary between processes. When the interpreter tries to acquire the GIL, the kernel simply does a context switch and queues the thread to the dedicated sync server. The thread is run normally by the sync server by popping it from the queue. When the GIL is released, another kernel call is made which simply switches the thread out from the sync server to a different core.

Since we are guaranteeing that synchronized code is running on a single core, it is the equivalent of a lock at the cost of a context switch. All other code can run in parallel on the remaining cores without problems. Moreover, this requires no changes to existing Python code or extension modules which acquire and release the GIL properly.

Obviously, the benefits from two cores will be negligible (possibly even negative because of the context switches and zero parallelism). However, every additional core you add will contribute to the parallelism of the process. Optimizing the context switches shouldn’t be too difficult because no memory pages are copied (no shared memory or IPC).

Limitations: The main limitation that I can foresee where scalability is limited by the sync server. The more code which needs to be synchronized, the greater the CPU usage on the sync core. Once that core is maxed out, the queue length may start increasing as the other cores wait around for the sync core. It looks to me that notwithstanding the cost of context switches this is at least as good as running everything on one core.

The core acting as sync server does not need to be special in any way. Neither does it need to be dedicated to the process alone. If the sync queue is empty (or even if its not), the kernel can always use the core for other processes like normal. The only guarantee we need from the kernel module is that the core will never be used for non-synchronized code within a process where it is designated as the sync server.

Does this make sense or am I crazy?

-Prateek Sureka

One key point I took away is that there is a big difference between the language and its implementation. Currently, the implementation of choice is CPython but there are many promising new projects like IronPython, PyPy and Jython (to which I am now a contributor) which hope to bridge the gap between Python and various other languages and fix some of the commonly raised complaints with the CPython implementation.

This is particularly relevant to Brainwave. Although we don’t expect to move to a different implementation anytime soon, there are some key areas which generated a lot of buzz in terms of lectures, open space talks and BoF sessions. The ones I’m specifically interested in today are 1) Security and 2) Parallelism

Python is an extremely introspective language. The dir() and eval() functions give you a lot of power to work with objects in ways the original creators never intended. This is a double edged sword. The problem is implementing security at the object level. The deep introspective nature of Python seems to be better suited to a capabilities based approach as opposed to a pure permissions model. Brett Cannon and I discussed a lot of options – running multiple instances of the interpreter as separate sandboxes in the same process space (not possible because of the way the CPython implementation caches object references) to adding keywords to the language (doesn’t seem like a good idea to change the language to serve the implementation). Brett is working on an interesting project to run a separate interpreter process in secure mode (very much like the old rexec solution, which, according to Brett, was dropped because of the number of bugs it caused in the implementation).

At any rate, you can always drop into C to get around all these obstacles (obviously not very pythonic!). It does mean that newcomers and enterprise developers will have a harder time getting to grips with some of these nuances. (Note: The IronPython and Jython teams get around this issue by leaving the security to the underlying VM – just like CPython).

Parallelism was another hot topic of discussion. .. and the guys at IPython are doing some really interesting things. The fundamental problem is with CPython’s global interpreter lock. Any thread in CPython needs access to the interpreter to run which requires acquiring a global lock. Hence, true parallelism cannot be achieved. It isn’t possible to eliminate the GIL because of the way CPython does garbage collection (reference counting). If
multiple threads had simultaneous access to the same objects, it would be pretty hard to track references across them. Stackless python (which is the first thing most people mention when this topic arises) solves the problem with concurrency but does nothing for true parallelism.
IronPython and Jython do not have a GIL because they once again use the underlying VM’s threading and GC models to implement true parallelism.

One good solution is once again to use multiple processes and either share objects between them (a la the POSH project) or do some form of RMI (Pyro, MPI4Py, PyMPI etc.)

Brainwave is currently using some of these workarounds. We have a capabilities based security model at the meme level. Each database query also carries with it information regarding the user context (i.e. who is logged in – effectively the users key). This is used at the end of the query to filter the result set of any protected data. Only accessible data is returned to the client. Since this data transfer is happening over the process boundary, it is harder (impossible?) for the client to subvert the server’s memory space.

The parallelism is currently implemented using Pyro. The engine is smart enough to use a host-process namespace so as to be able to identify all processes across all hosts (multiple processes can run on the same host to take advantage of multiple cores/processors). The code is still under heavy development so I don’t have all the answers yet regarding exactly what algorithms are being used for the multiple master transaction based data synchronization. We expect to prototype a few options (let me know if you have suggestions) and go with whatever has the best benchmarks.

I’m going to keep an eye on what the CPython guys come up with to deal with these issues.

-Prateek Sureka

JUnpickler: This is a Java based unpickler for Python pickle data. The idea is that you can serialize some data in Python and then get the string into Java and deserialize it to get a Java object.

The code is released under the Eclipse Public License v1.0 (EPL). It’s one of the least restrictive licenses I could find.

Please let us know what you think or if you want to contribute.

Prateek Sureka

I found people teeming with optimism. Looking towards the stars – proud and egotistical about the future of the country. But something seemed amiss. The rest of the world still looked at India as a land of cheap labor, annoying tele-marketers, customer service reps and gut wrenching curry.

Yes we’re smart. Yes we’re enterprising. So why wait for the wage arbitrage to run out? The time was ripe to create an institution that was committed to technical excellence. Why not create our own “insanely great” products? But first you need a better hammer and chisel. I was back soon. Inspired by those who go against the grain to challenge the assumptions, affect the masses and make a difference, Brainwave was founded on the following principles.

• product centric

• design oriented

• technology driven

• mecca for developers

The Sureka Group was perfectly positioned to take advantage of this opportunity. Cash rich and ready for a challenge, the group was full of encouragement. Enthused with its success in manufacturing and real estate, its combined experience has been a boon to this fledgling enterprise with lofty goals.

And so we begin. As we proceed through this endeavor, we promise to always…

• deliver cutting edge products

• tackle tough technical challenges

• never cease to amaze

- Prateek Sureka