Saturday, June 28. 2008
Brainwave Platform 1.2 released
Thursday, April 3. 2008
How we stack up against Developer requirements!!
A link was forwarded to me the other day on "A developers Bill of Rights" and is available on "http://www.spoiledtechie.com/post/My-Bill-of-Rights-as-a-Programmer.aspx"
It set me thinking of how much we score and not surprisingly we came out pretty well (over 85%...i know it implies there is scope for improvement, but then even perfection can be improved, so am not really complaining too much right now.)
Well, here is how we stack up
1. Every Programmer shall have a quiet working environment.
Almost Always
2. Every Programmer shall have the right to listen to music.
We provide free headphones so as to not be forced to listen to someone else music!
3. Every Programmer shall have a fast PC.
3 GHZ, 2 GB RAM, with atleast dualcore/core2 duo machines, MAcbooks, Macminis, we are better than most other companies.
4. Every Programmer has their choice of both Keyboard and Mouse.
Well as no one has any objections as yet, so i feel yes.
5. Every Programmer shall have a fast Internet connection.
This is an area where we have always upgraded but can always further improve.
6. Every Programmer shall have two or more monitors.
Yes. Since the very beginning we have done this!
7. Every Programmer shall have a high back comfortable chair.
We are giving Mid-high back chairs for all devs
8. Every programmer shall have access to the right tools.
We allow people to download and use any s/w tool they want.
9. Every programmer shall have the right to have admin access on the PC.
Almost all confirmed employees have all required access.
10. Every programmer shall be allowed to attend a developers conference once a year.
Some of the devs have been able to attend, but so far mainly marketing folks. So this is an area of improvement.
So maybe we can give ourselves a small pat on the back and plan ahead!!
Thursday, March 20. 2008
Brainwave presentation @ BarCamp5
As promised, we are happy to share with you, the presentation slides that Prateek used at Barcamp5, Hyderabad to talk about our meme based, schemaless, application development platform.
Hope you enjoyed the slides. Please feel free to add your comments/ observations and suggestions below.
Cheers!
Abhishek
Friday, February 15. 2008
Brainwave @ barcamp5, Hyderabad
Prateek and Vinod are going to be representing Brainwave at Barcamp5 to be held in the Google campus at Hyderabad on the 16th Of February,2008. You can catch them there making a 20 minute presentation.
Pics and more updates will soon be posted in the brainwave blog. You can also check out http://barcamp.org/BrainWaveLive and http://barcamp.org/BarCampHyderabad5 for details on the event.
Look forward to meeting you all there,
Regards,
Abhishek Chamaria
Wednesday, September 12. 2007
Suggestion for removing the Python Global Interpreter Lock
This is a hard problem (as evidenced by Greg Stein and Mark Hammond?'s efforts in 1999) mainly from a performance point of view. GvR himself has stated that although he doesn't have the time to do it himself, he would be willing to accept any patches to CPython which removes the GIL (as a compile time option) which does not negatively affect performance of single threaded applications and significantly improves I/O bound multi-threaded apps.
As I learnt at PyCon 2007, the problems in general have to do with Python's garbage collection mechanism which uses reference counting and hence requires synchronized access to objects to ensure that the increfs and decrefs are done properly. Additionally, the GIL protects various caches (notably the integer (and some other types?) free lists so that creating and destroying those items don't take too much time because the PyObjects are recycled.
So far, the solutions discussed involve some type of fine grained locking around critical code sections. The problem here is that beyond two or three cores, this doesn't scale well because of lock contention (since so much stuff needs to be synchronized).
Well, what if we tried a different approach... Locks don't seem like they're going to solve the problem in a scalable way and if we're going to take the effort to do this, might as well do it so that the benefits scale in an environment of more than 100 cores (its possible - see Azul Systems).
What if we dedicated one core for running synchronized code? Naturally this can't be done without support from the kernel (perhaps some sort of kernel module/VxD).
At process initialization time, the kernel dedicates a single core to be the "synchronization server" and provides an instruction queue for it. This can vary between processes. When the interpreter tries to acquire the GIL, the kernel simply does a context switch and queues the thread to the dedicated sync server. The thread is run normally by the sync server by popping it from the queue. When the GIL is released, another kernel call is made which simply switches the thread out from the sync server to a different core.
Since we are guaranteeing that synchronized code is running on a single core, it is the equivalent of a lock at the cost of a context switch. All other code can run in parallel on the remaining cores without problems. Moreover, this requires no changes to existing Python code or extension modules which acquire and release the GIL properly.
Obviously, the benefits from two cores will be negligible (possibly even negative because of the context switches and zero parallelism). However, every additional core you add will contribute to the parallelism of the process. Optimizing the context switches shouldn't be too difficult because no memory pages are copied (no shared memory or IPC).
Limitations: The main limitation that I can foresee where scalability is limited by the sync server. The more code which needs to be synchronized, the greater the CPU usage on the sync core. Once that core is maxed out, the queue length may start increasing as the other cores wait around for the sync core. It looks to me that notwithstanding the cost of context switches this is at least as good as running everything on one core.
The core acting as sync server does not need to be special in any way. Neither does it need to be dedicated to the process alone. If the sync queue is empty (or even if its not), the kernel can always use the core for other processes like normal. The only guarantee we need from the kernel module is that the core will never be used for non-synchronized code within a process where it is designated as the sync server.
Does this make sense or am I crazy?
-Prateek Sureka
Tuesday, March 20. 2007
Random Thoughts:Quality of programmers & Microsoft
There is another menace that is rapidly decreasing the number of true programmers. I personally believe that it is a very bad idea to provide a developer with a framework unless the framework facilitates and bolsters application performance and ease of use to the end-user. Typically a framework stops the developer to think beyond the framework because almost everything is provided by it. He does not have to worry about cookies, sessions, functioning of the web server, database security or the innards of the OS. A developer becomes narrow-visioned who cannot think of anything but what is provided to him by the framework. In the next 3-5 years RAD tools and Frameworks are going to convert a developer into an high end user. They wont even bother to know what the OS they are working on, what protocol they are using to communicate with the server, what is a server, how things work under the hood as they do not need to know. Everything is on their platter.
Friday, March 2. 2007
Pycon 2007 Lessons on Security and Parallelism
Pycon 2007 was a great experience. I got to meet Guido and heard him speak on Python 3000 which I thought was quite illuminating.
One key point I took away is that there is a big difference between the language and its implementation. Currently, the implementation of choice is CPython but there are many promising new projects like IronPython, PyPy and Jython (to which I am now a contributor) which hope to bridge the gap between Python and various other languages and fix some of the commonly raised complaints with the CPython implementation.
This is particularly relevant to Brainwave. Although we don't expect to move to a different implementation anytime soon, there are some key areas which generated a lot of buzz in terms of lectures, open space talks and BoF sessions. The ones I'm specifically interested in today are 1) Security and 2) Parallelism
Python is an extremely introspective language. The dir() and eval() functions give you a lot of power to work with objects in ways the original creators never intended. This is a double edged sword. The problem is implementing security at the object level. The deep introspective nature of Python seems to be better suited to a capabilities based approach as opposed to a pure permissions model. Brett Cannon and I discussed a lot of options - running multiple instances of the interpreter as separate sandboxes in the same process space (not possible because of the way the CPython implementation caches object references) to adding keywords to the language (doesn't seem like a good idea to change the language to serve the implementation). Brett is working on an interesting project to run a separate interpreter process in secure mode (very much like the old rexec solution, which, according to Brett, was dropped because of the number of bugs it caused in the implementation).
At any rate, you can always drop into C to get around all these obstacles (obviously not very pythonic!). It does mean that newcomers and enterprise developers will have a harder time getting to grips with some of these nuances. (Note: The IronPython and Jython teams get around this issue by leaving the security to the underlying VM - just like CPython).
Parallelism was another hot topic of discussion. .. and the guys at IPython are doing some really interesting things. The fundamental problem is with CPython's global interpreter lock. Any thread in CPython needs access to the interpreter to run which requires acquiring a global lock. Hence, true parallelism cannot be achieved. It isn't possible to eliminate the GIL because of the way CPython does garbage collection (reference counting). If
multiple threads had simultaneous access to the same objects, it would be pretty hard to track references across them. Stackless python (which is the first thing most people mention when this topic arises) solves the problem with concurrency but does nothing for true parallelism.
IronPython and Jython do not have a GIL because they once again use the underlying VM's threading and GC models to implement true parallelism.
One good solution is once again to use multiple processes and either share objects between them (a la the POSH project) or do some form of RMI (Pyro, MPI4Py, PyMPI etc.)
Brainwave is currently using some of these workarounds. We have a capabilities based security model at the meme level. Each database query also carries with it information regarding the user context (i.e. who is logged in - effectively the users key). This is used at the end of the query to filter the result set of any protected data. Only accessible data is returned to the client. Since this data transfer is happening over the process boundary, it is harder (impossible?) for the client to subvert the server's memory space.
The parallelism is currently implemented using Pyro. The engine is smart enough to use a host-process namespace so as to be able to identify all processes across all hosts (multiple processes can run on the same host to take advantage of multiple cores/processors). The code is still under heavy development so I don't have all the answers yet regarding exactly what algorithms are being used for the multiple master transaction based data synchronization. We expect to prototype a few options (let me know if you have suggestions) and go with whatever has the best benchmarks.
I'm going to keep an eye on what the CPython guys come up with to deal with these issues.
-Prateek Sureka
Wednesday, December 20. 2006
Resume Writing can make or break your image.
Hi all,
Its interesting to know how candidates make life difficult for themselves while writing their resumes. Imagine getting applications where the candidate has all the requisite skills and has not mentioned his/her contact information! To me it implies that the candidate does not have a high regard for attention to detail. It also implies that the candidate neither thought before starting the resume writing process nor did (s)/he bother to check for completeness before posting the same online.
Some other interesting observations that I think will help any candidate applying for a job across the world are listed below in the order we discovered them while sorting through the resumes received by us at Infocom 2006 and subsequently after candidates decided to take the challenge on our website and uploaded their resumes.
Some of the common Turn-Offs
- Contact details missing....well, does the candidate feel so sure of rejection!
- Incorrect contact details....duh, it is as bad as the previous one or maybe even worse!
- Just mentioning years of experience without giving any details of the kind of experience...makes me want to know if the candidate feels that his experience is irrelevant to the position being applied for!
- All candidates from an University stating their Dean/Director as a reference...wonder how many candidates will the dean personally know!
- A candidate felt that mentioning the reasons of quitting his previous organization should be part of the resume...I guess, I do not want to hire somebody who wants to talk bad about his past employer in an resume!
- A couple of candidates actually mass emailed their resumes to about 35 different companies and addressed it all to just 1 of them....as you may have guessed, it was not even addressed to my organization, so should I be processing the resume?
- This one takes the cake; I got 1 resume that was typed in all caps...HELLO! ARE YOU SHOUTING AT ME or applying for a job?
- 1 candidate is probably a wannabe model, 50% of the 1st page is his photograph....looks and muscles may be important but I think we are recruiting for brains rather than brawns, or was that pose meant to intimidate me?!
- Another amazing one is from a candidate who has 3 years of experience working on Python 4.2.2 ....hmmm, when did this release!
Wonder what kind of experiences other people have when it comes to sorting through resumes. I would like to expand the above list for the benefit of all candidates, so am looking for contributions from all the readers of the post.
Cheers!
-Abhishek Chamaria
Wednesday, December 20. 2006
Infocom 2006 Photographs are up!
Hi all,
As promised we have uploaded some images from Infocom 2006 on our website. Do check them out at http://www.brainwavelive.com/company/events.html
We will soon upload a video of Prateek, the whiz whose brain runs the technology behind brainwave, addressing the delegates at Infocom 2006 on the Innovative trends in Database Management systems.
Cheers!
-Abhishek Chamaria
Thursday, December 14. 2006
Brainwave Code Repository
I've been working for a while on a code repository where we can put up our open source projects and get some community participation going and I think we've finally got something...
JUnpickler: This is a Java based unpickler for Python pickle data. The idea is that you can serialize some data in Python and then get the string into Java and deserialize it to get a Java object. The page is up at http://code.brainwavelive.com/unpickler
The code is released under the Eclipse Public License v1.0 (EPL). It's one of the least restrictive licenses I could find.
Please let us know what you think or if you want to contribute.
Prateek Sureka
Wednesday, December 13. 2006
Developer Preview Launch - (Recruitments and follow up)
Hello and welcome back friends,
All in all Infocom seems to have been a great launching pad in India. We had to reorder the developer preview kit CDs and promotional material to cope with the demand. A lot of interesting feedback was received and now we need to start following up on all the contacts generated through the event.
Today was an hectic day. I was trying to sort through the 400 odd resumes we received at Infocom 2006 and also had my team collate the various responses, comments and feedback received from the developer community and end users.
As a policy we have 4 rounds of screening:
- Online Aptitude test
- Matching the Candidates skill sets to our Requirements (based on the resume)
- Telephonic Interview
- A round of orientation and up to 3 personal interviews along with team based activities.
Some of the key personality traits that are important to us are:
- An inquisitive mind open to research and learning
- A sense of team spirit
- A keen sense of humor
- Willingness to shoulder and fulfill responsibilities
- Desire to go the extra mile and ensure that the work is not just completed as per expectations but is actually exemplary
- Laziness - This ensures that the code you write will be lean and mean and always reusable!
- Focus and commitment
On our side we know that if you dole out peanuts you will at best get monkeys! So we ensure that employees have:
- A healthy work life balance, have a comfortable work environment,
- Users have 2 monitors instead of the usual 1 to maximize screen real estate.
- Flexi time is the norm.
- Free lunch is provided to all employees on Saturdays.
- We believe in constant improvement and learning throughout life so we play videos from the leading universities/ websites and have regular training sessions.
- Meeting milestones implies that the employees get time off to relax, unwind and even play games on the Xbox against each other on the projector!
- We believe in the 70:20:10 rule and all employees get 10% of their working time and company resources to follow their dream projects!
Now I guess its time for me to get back to inviting the short listed candidates to take the test on our site. If you are one of them then I would like to wish you all the best! If not then no worries, it was a pleasure to have you come to my weblog and your comments and feedback are welcome.
Cheers!
Abhishek Chamaria
Tuesday, December 12. 2006
Developer Preview launch - Infocom 2006
Hello friends,
After operating in stealth mode for nearly a year, I am happy to announce the Indian launch of the 'Brainwave Platform' at Infocom 2006 in Kolkata, India. The first release is a developer preview. Since a part of our development team is based in Kolkata and it is an upcoming IT destination, we thought it would be appropriate to start here.
Infocom 2006 was held at the Salt Lake Stadium from December 6-10, 2006. Brainwave was the brand sponsor for the Innovation Hub, a pavilion that showcased the top 30 innovations made by students across the globe. Also represented at the show were the folks from Intel, Siemens, Wipro and others.
Preparing for the launch was gruelling but fun. We all made many mistakes and learnt from them. The team camped at the venue for a few days with the contractors to ensure that the Brainwave booth was ready on time. Inspite of some technical and logistical problems, we got the job done.
Our Technology Architect and founder, Prateek Sureka addressed the industry leaders as a speaker on December 8, 2006. He spoke on the shortcomings of relational and object oriented database models and highlighted some of the research trends. He also said a little about our own patent pending information architecture.
Overall the event was a success and had people from various parts of the globe attending. We received some great reviews, coverage in the local media, accolades on our stall design and generated a number of enquiries. We got great feedback on Prateek's presentation and the live demo applications created in under 30 minutes by the dev team manning the booth. My Q&A sessions in the booth were also very productive and I hope I was able to whet the appetite of potential employees, development partners and end users alike.
It would be unfair on my part to not talk about the excellent efforts put in by some of our India development team and Marina who had flown down specifically from the US for the event. I will be posting those as well as some details of our lives during the launch, images, scanned articles and what we learned from our experiences as a subsequent post. Look forward to all your comments and feedback.
Cheers!
Abhishek Chamaria
Tuesday, December 12. 2006
My First Entry
I had just returned from Peru after a trek to Machu Picchu to find India had changed since I left five years ago. College in the United States had been an amazing experience and I was set to return for a stint at a fast growing technology company on the East Coast. But in the few months I had before I started work, I set forth to rediscover my homeland.
I found people teeming with optimism. Looking towards the stars - proud and egotistical about the future of the country. But something seemed amiss. The rest of the world still looked at India as a land of cheap labor, annoying tele-marketers, customer service reps and gut wrenching curry.
Yes we're smart. Yes we're enterprising. So why wait for the wage arbitrage to run out? The time was ripe to create an institution that was committed to technical excellence. Why not create our own "insanely great" products? But first you need a better hammer and chisel. I was back soon. Inspired by those who go against the grain to challenge the assumptions, affect the masses and make a difference, Brainwave was founded on the following principles.
- product centric
- design oriented
- technology driven
- mecca for developers
And so we begin. As we proceed through this endeavor, we promise to always...
- deliver cutting edge products
- tackle tough technical challenges
- never cease to amaze
Calendar
|
July '08 |
| Mon | Tue | Wed | Thu | Fri | Sat | Sun |
| 1 | 2 | 3 | 4 | 5 | 6 | |
| 7 | 8 | 9 | 10 | 11 | 12 | 13 |
| 14 | 15 | 16 | 17 | 18 | 19 | 20 |
| 21 | 22 | 23 | 24 | 25 | 26 | 27 |
| 28 | 29 | 30 | 31 |
Quicksearch
Categories
Syndicate This Blog
Blog Administration
Powered by
