John Lam on Software

Dynamic Languages at TechEd 2009

Dynamic languages on .NET are picking up momentum at this year's TechEd. Your typical TechEd attendee is a mainstream .NET developer, since this conference focuses on technologies that are shipping today (as opposed to futures conferences like the PDC). To speak more to this crowd, I focused on how they can make their existing .NET apps better by mixing in some end-user scripting.

My talk was DTL332 if you're a TechEd attendee (not sure when/if the videos will open up to the general public). I showed how you can add Ruby and Python scripting to an existing app, and spent some time building some simple REPLs. Toward the end of the talk, I showed a more realistic scenario where I embed a REPL in an existing Open Source .NET application: Witty.

During the demo, I showed off a twist on the traditional REPL - one where the editor and the REPL are one and the same. To run code in this REPL, you select the code that you want to run, and the output appears immediately below the selection. Since the REPL is just an editor, you can just delete any output that you don't want. History behaves just fine as well. Here's a screenshot of the REPL in action:

There are a couple of other notable things in this REPL. First, it syntax colors the text that ran correctly. Second, it supports more than one .NET language - you can switch between Python and Ruby using %python and %ruby (although whether that's a good idea is left to the reader). Third, all of its configuration is self-contained inside of a single DLL. This means that all you need to do to add scripting support for your app is add a reference to a single assembly. The assembly will write all user configuration files to well-known locations on disk (within your HOME directory) the first time it is run.

This idea of a REPL within an editor buffer isn't new by any means. I've been spending some time hanging out with Mark Hamburg recently, and this is his favorite way of implementing a REPL (he's done quite a few, including the embedded IDE that the Adobe Lightroom folks use to build Lightroom). He won me over on the idea, so I thought that I'd hack up an implementation for folks to experiment with.

I pushed the code up to Github as the repl-lib project, so feel free to fork, play with the code, and send me some feedback on the idea. Note that there's a nice long tail of bugs in this REPL, so feel free to fix bugs and submit changes!

Here's a screenshot of the Witty main window; notice the console button:

Here's a sample console session:

The last line of code causes the Witty UI to display:

After poking around inside Witty, I think there is some work that needs to be done to make scripting truly a first-class citizen in the project. While some might look at this as bad news or an indictment of Witty, it really isn't. Any project that isn't designed for such an invasive feature will likely make lots of assumptions that make it hard to make that feature successful.

But the good news here is that if you use scripting as a end-user feature to drive your refactoring efforts, you will wind up with an app that is easy to script, but more importantly, is better designed. I argue that your design is better because it's forces a clean separation of concerns in your code: especially your models and your views. This separation of concerns will make it easier to test your end-to-end scenarios and script your integration tests. This all adds up to better software for everyone.

I've made it pretty easy to add scripting support to your app. Grab the repl-lib project and spend a few minutes (that should be how much time it takes) to integrate scripting into your project. If you're interested in how much work it took to integrate these changes into Witty, check out this diff; most of the stuff is VS-related, and ReplResources.xaml is no longer required in current builds of repl-lib:

Disable that annoying beep on Thinkpads when resuming from Sleep

It’s under the Power Manager settings:

Hello, 2009

Happy New Year!

It’s been a hectic holiday season here in the Pacific Northwest. We were snowed in over here for somewhere in the neighborhood of 10 days or so. I wanted to spend most of my time with family, but I’m sneaking away for a few hours today to prepare for the new year.

I’m writing this post from my office in my new chair that I picked up from Plush over at the Redmond Town Center:

No that’s not me :) It’s a fantastic chair. I wanted an alternative place to work that wasn’t just my desk (which is a great setup, but a bit of variety is a good thing). I got the fur cover which makes it feel like I’m sitting on a giant teddy bear.

I’m writing this post on my Lenovo X200s, which finally arrived after deciding to take a tour of Canada for a week:

This is a fantastic laptop. The keyboard is hands-down the best keyboard that I’ve used. It’s significantly better than the T61p that I have, and I’m finding that I prefer the feel of its keyboard over my 17” MBP as well. I’m loving the fact that I now have the extra keys that I’ve been missing on the Mac. Form Follows Function.

My top criteria around laptops are:

1) Screen quality – this thing has a 1440 x 900 LED backlit display which is fantastic. A good compromise considering the laptop weighs in at around 3.2 lbs with the 9-cell battery that I have installed.

2) Battery life – this thing has crazy battery life. I have the screen set at max brightness right now, and my battery meter shows 7 hours remaining at 88% charge.

3) Emissions – heat and sound. This thing is quiet and is not hot at all. This was the thing that annoyed me the most with the MBP – it was hot *and* loud. Playing a movie on it without headphones on was an exercise in frustration.

4) Performance. This thing is plenty fast with the 1.86GHz Core 2 Duo. It rates 4.9 on the Vista CPU perf, but that doesn’t tell the whole story. It has 6MB L2 Cache, but I also paired the laptop up with the most excellent Intel X25-M SSD.

The Intel SSD was the thing that turned this into a very special laptop. There are no annoying pauses at all while working. My system is instantly responsive after logging on – which certainly isn’t the case with my quad core desktop because of all of that contention for the HDD head. It boots really quickly, shuts down fast.

Now that I’m all geared up for the new year, I’m looking forward to the goals that I’ve set for myself:

1) Ship IronRuby.

2) I want to make it just as easy for an external developer to work on IronRuby as someone who sits down the hall from me.

3) Work on the fit and finish that will make IronRuby a great experience for .NET developers. Integration with ASP.NET MVC, fixing our interop bugs, fixing our startup and working set issues, and getting some quality docs out there.

4) Refactoring me. I’m going to fix a bunch of things that I’m not happy with. I’m turning into a fatso again and that’s not cool. Part of this is admitting that I have a problem, and then figuring out that I don’t have to do it alone. I’m not going to do crazy things like my brother is doing (40” vertical in 2009!) but I should be able to hit 150 lbs this year.

I hope you all have a great year!

PDC 2008 Wrap-up

As usual, PDC 2008 was a great event. I had a chance to meet a lot of old friends (it’s been 7 years since I’ve been to a PDC- it attracts a certain crowd that only shows up there) and I’m happy to see that everyone’s doing well.

My IronRuby talk was the first day of the conference. It was a crazy mixture of many, many demos (9 in all) which showed:

- when and why you should create a new type in IronRuby using C#

- how to write a Visual Studio plug-in using Ruby

- how to build a unit test and mock object framework

- how to integrate Ruby scripting into your existing C# application

- how to build simple web services using Sinatra

Only one blew up (the last one) when I forgot to change the type name in my ironruby_mischief.rb file to the type that I created in the IronRuby libraries.

You can watch it on TV now (Silverlight required).

Brief Rant:

Session evaluations are a great way to give direct feedback to the teams. Sure, you can send email to the speakers directly, but session evaluations get circulated fairly broadly internally and are often used by management for future product plans. They are a great way for you to influence the way we plan and prioritize the work that we need to do.

But there’s really not a lot in it for you, the attendee, outside of maybe some personal satisfaction that you helped to influence the products that you use (that’s if you knew how influential they can be in the first place).

Here’s my proposal:give direct incentives for folks to fill out their evaluations. It can be something as simple as 1 eval == 1 ticket for a drawing for prizes at the end of the conference. The more evals you submit, the greater your chances for winning prizes that are donated by the sponsors.

You could do things like what Stack Overflow does: give merit badges to folks that provide quality feedback. Written comments are always preferable to just clicking on radio buttons, so you could earn more merit points by doing just that. The whole system is online anyway, and it wouldn’t be that hard to implement.

You could even do something simple like write 5 evaluations to get your conference T-shirt. That should significantly improve the response rate, considering what the awesome power of T-shirts are in the geek world :)

I don’t believe that this would skew the feedback in a significant way. Hopefully we’ll hear more from the ‘silent majority’ this way. For example, I had 245 people in my IronRuby talk and as of this writing, I only have 15(!) evaluations.

If you read this far (and you attended my talk at PDC) please click here to submit your evaluation. Apparently that link is some kind of one-time link. You'll (unfortunately) have to go to the PDC site and navigate to my session (TL44).

What I liked:

I loved the Big Room. It was a place where folks could gather to meet and mingle. Since I want to talk to customers at the show, I was happy to spend virtually all of my time in the Big Room talking to customers.

But there was one talk that I did attend: Miguel de Icaza’s awesome Mono and .NET talk. Horrible title – it really should be called “Awesome Mono hacks by Miguel and his band of merry hackers”.

Here’s the coolest thing that he showed:

Mono applications running on a non-jailbroken iPhone. Yes, you heard that right. For those of you who aren’t in the know about these things, the iPhone SDK prevents you from writing a JIT compiler through some kernel restrictions (you can’t mark a writable page as executable). It also prohibits you from writing an interpreter so that they can maintain their lock on application distribution via the App Store. So how did they do it? They made it possible to compile an entire Mono application into a single binary executable which gets signed via xCode and downloaded to the phone. This opens up some pretty awesome opportunities for using .NET as a platform for building apps that run on the iPhone.

He also showed a very cool C# REPL in action. There are some cool UI ideas from his REPL that are going to find a nice home in the IronRuby REPL :)

Lots of fun banter during the talk as well. Click here to watch it on TV (video not there right now at the time of this writing, but should be there soon).

I had a chance to meet a lot of folks, and once my talk was done, I decided to try and shoot photos of as many of my friends as I could. I’ll continue to upload them to my PDC 2008 flickr photoset as I process them on my laptop.

More later as I continue to collect my thoughts.

The Fukuoka Ruby Delegation

I had the honor of meeting with a delegation from Fukuoka, Japan this morning at Microsoft's Executive Briefing Center. Part of the delegation represents the Ruby Business Commons, one of the largest Ruby SIGs in Japan (> 500 people as of August 2008). They were here to talk to us about Ruby, its impact in Fukuoka, and how we can work together to help promote Ruby and IronRuby in the IT industry in Fukuoka.

It was a challenging meeting since everything had to be done through a translator (who did a remarkable job). They had some great questions and suggestions for how we can work together to help promote Ruby in the Enterprise in Japan. I'm looking forward to seeing what we can do together to help spread the love.

My Dev Kit

Update: Added missing profile.ps1 file.

The IronRuby team is getting ready for another major push for the PDC. To get ready for this (be careful what you wish for!) I'm upgrading my laptop's hard drive from the stock 160GB that it came with to a brand-spanking new Black Scorpio 320GB drive.

This also means that I'm repaving my Boot Camp partition (I had previously allocated a measly 40GB on the old drive). While waiting for software to install, I figured that folks out there might be interested in what my setup is for a computer. So, here it is:

Operating System and general productivity tools:

I run Windows Vista SP1, 32-bit. I had tried to get Windows Server 2008 64 bit installed in the BootCamp partition, but it was a lost cause. The combination of my pre-Santa Rosa 17" MBP hardware and the general lack of 64 bit drivers made it virtually impossible to do a native OS install on the Boot Camp partition. I should be able to get 64 bit Windows Server 2008 installed in a Fusion virtual machine. I'll report later how that goes.

I do have a mildly pimped out Vista Sidebar. I'm a huge fan of Christian's widgets. I use his CPU, Memory, and Network Traffic meters. I run WeatherBug, and Drive Activity as well. I don't find it nearly as useful on my laptop as I do on my desktop where I have two 21" panels.

I use Internet Explorer quite a bit, especially for browsing internal Microsoft sites. I leave it set as my default browser, since I've always loved its startup time vs. Firefox. I'm currently using IE 7, but I do have IE 8 Beta 2 installed on one of my desktops at the office.

I use Firefox 3 quite a bit as well, especially since it has Vimperator, which adds VI key-bindings to Firefox. IE should seriously consider building a Firefox compatible plug-in API so that it can run Firefox plug-ins.

I'm a big fan of PicLens, and I have it installed on both IE and Firefox.

As you might imagine, I spend a lot of time in Outlook 2007. While I have a hard time understanding why it needs to use 36 threads and open 3,319 OS handles to do its thing, it works quite well once I've pimped it out a bit with my three basic macros.

I use Windows Live Writer to compose all of my blog posts. It works great with TypePad, who is my blogging hosting provider. Life is too short to run your own blog engine on some random hosting provider (I used to use Dreamhost back in the day). I want it to be somebody else's problem to solve if my site goes down.

I use GridMove as a tiling window manager using its 2-part grid vertical template. CTRL-ALT-1 moves the active window to the left *half* of the screen, and CTRL-ALT-2 moves the active window to the right *half* of the screen. On my desktop, CTRL-ALT-3 moves the active window to the left half of monitor 2 and CTRL-ALT-4 moves the active window to the right half of monitor 2. It's easily the most useful utility that I've added in a long time.

Developer Tools

I use Visual Studio 2008 Team System, SP1. I configure it via a custom vssettings file that I've carried around from machine to machine.

I use multiple enlistments so that I can do work in parallel. Some of the devs on my team have 6 enlistments(!) Such is life when you have to battle the Troll to get your changes committed into the source repository.

I use ViEmu to enable VI key bindings in Visual Studio. Developer Division at Microsoft has a site license for it, but I'd definitely pay my own money for this.

My primary text editor is vim. I have a custom .vimrc file that I've carried around from machine to machine. It's not as pimped out as some other folks' vim setup, but it works for me.

I use Windows PowerShell for my day-to-day work. I have a mildly pimped out PowerShell console that uses a directory that I cloned from Scott Hanselman's machine. Like a lot of folks, I can't work without a good set of aliases. Those can be found in my profile.ps1.

I have a Tools directory that I carry around from machine to machine. There's lots of stuff in it, but here's a general breakdown of the tools I use most often:

• Keith Brown's Password Minder
• Most of Mark Russinovich's Sysinternals Tools
• Notepad 2 - it's awesome for viewing integrated diff files.
• Lutz Roeder's Reflector
• Beyond Compare 2.0
• AGREP
• RemapKey from the Windows Server Resource Kit (I remap Caps Lock to CTRL via this, reverse the Windows / ALT key mapping on the left side and re-assign my right Command key to DEL).

IronRuby uses Subversion for our external-facing source control. Our SVN repo lives on RubyForge, and I use a combination of Tortoise SVN and the command-line tools to do my day-to-day work.

IronRuby is using GIT for a few of our sub-projects. Eventually we'll be moving the primary source repository from SVN to GIT. Like most folks using GIT these days, we use the awesome github service. I use the MSysGit implementation on Windows with the option to run GIT from the Windows Command Prompt enabled.

The Cloud

More and more of my life is moving to the cloud. Here's a list of the services that I use, and the places where I go.

I use flickr for posting my photos. I've been a Pro member for quite some time. For $25 / year it's a bargain. I don't really use flickr as a social network like some other folks, but I'm quite happy with how well it works for me.

I use twitter to post random bits of status. The 140 character limit encourages posting. I'm pretty much only using the blog to post longer pieces.

I use Google Reader to consume my RSS feeds. I use multiple computers every day, and I like the fact that read / unread status is stored in a central location.

I use Google Apps for my domain for my personal email. I used to run my own Exchange Server back in the day, and the day I switched was an awesome liberating moment for me. I no longer run any servers in my house.

I have a facebook account that I don't use all that much. It's pretty much at the bottom of my list of places to go. My status updates are via twitter, which makes it look like I'm on facebook a lot more than I actually am.

What about you?

As you might have figured out by now, I'm a big fan of minimalist UI, black backgrounds and keyboard-driven UI's. What's in your dev kit, and why do you love it?

IronRuby at OSCON

OSCON is a happy place for me. It’s my third OSCON, and I created good memories at each one. At my first OSCON, I was showing off RubyCLR to the world, and I met some folks from Microsoft who convinced me that it was the right time in my life to go ship something big. At my second OSCON, I was working for The Man, and we announced that were accepting contributions into IronRuby – a big change at the time.

Today, we’re raising the bar even higher in IronRuby. There are three changes happening today:

1. We’re shipping our first binary release. In this package, we’re taking a “batteries included” approach and shipping the Ruby standard libraries in it. This takes us a big step closer towards “IronRuby runs real Ruby programs”.

2. We’re announcing a new project: ironruby-contrib. It’s hosted at GitHub, an awesome place to do collaborative development. The goal of ironruby-contrib is to provide a place where folks from the community can participate in projects that enhance IronRuby or its underlying platforms. The first project in ironruby-contrib is the Rails plugin that we demonstrated at RailsConf this year. It makes it easier for Rails developers to add Silverlight to their Rails applications, and contains some of our ideas about migrating controller code to the client. Jimmy Schementi is the brains behind the project; see his introductory blog post for more details.

3. We’re announcing that we’ve submitted our first set of changes to the awesome RubySpec project. We’ve been talking about contributing to RubySpec for a long time now, but various things kept us from doing so. We’ve fixed those things now, which takes us a big step closer towards being an active participant in all things Open Source, and not just the things that we create.

IronRuby and ASP.NET MVC

At Tech Ed last week, I showed a prototype of integration between IronRuby and ASP.NET MVC. I was really happy about how straightforward this turned out to be; it’s a strong testament to the robustness of our hosting API that Levi on the ASP.NET MVC team was able to add IronRuby integration with only a simple example and the existing docs.

Let’s walk through a simple example. Keep in mind that this is a prototype, and that many things will likely change before we’re done with it (particularly some of things we do today with globals and instance variables).

First, let’s look at how you can define some default routes via routes.rb:

Next, let’s look at the Controller code for the HomeController:

The index action renders a view using a trivial Model which lives in models\HomeModel.rb:

This model is used by the index action View, which lives in views\home\index.rhtml:

Next, let's look at a slightly more sophisticated example using a Products Controller that renders a list of categories that looks like:

It also renders a list products for a category when you click on it:

This is what the controller looks like:

There are two actions in this controller: index and list. The index action returns a list of the categories, and the list action returns a list of products for that category.

Note that we're using instance variables on the controller to communicate with the view - this is likely going to change in the future, but it was something we hacked in to see what it would look like.

If we look at ProductsModel, you'll see that it contains a simple adapter around a Visual Studio generated wrapper for the Northwind database:

We had to write our own adapter to work around a few bugs that we have in our .NET interop story as of right now, as well as adding a few model-esque methods. Here's what the NorthwindDatabase adapter looks like:

Notice how we can use LINQ to SQL to write our queries to the database, and the funky C# 3.0 anonymous delegate syntax that we use for GetCategory().

The interesting code in the view looks like:

Today we're using the MVC template engine, but there isn't any reason why we couldn't enable erb / haml etc. in the future.

How can I get it?

You can download a ZIP of the entire project from here if you want to take a closer look. Note that you can't run these bits yet, since I demo'd it using a private build of ASP.NET MVC from mid-way through their current milestone. That said, when Preview 4 of ASP.NET MVC ships you'll be able to run IronRuby on top of it (their current release is Preview 3).

Look for a blog post here, and from the folks like Phil over in MVC-land when we're ready to ship you some bits to build MVC apps using IronRuby.

IronRuby at Tech Ed 2008

I’d like to start out by thanking everyone who came up to talk to me at the conference. We had a Dynamic Languages booth at the conference, and it was great to talk to all of the folks who took the time out to drop by for a visit.

A big thanks goes out to our booth staffers – Dave Fugate, Curt Hagenlocher, Jimmy Schementi, Bill Chiles and Mahesh Prakriya who helped to keep things running smoothly. Dave somehow managed to grab an enormous whiteboard from somewhere and used it to help answer many of the FAQs for folks who were hanging around waiting for a blue-shirted guy to become available.

Tech Ed is (not surprisingly) a very enterprisey conference. We spent at least as much time telling folks what Dynamic Languages were (or correcting their misperceptions) as we did talking about our team’s progress. We also had a whole bunch of folks come up to ask us questions about F# because, apparently, F# is now a dynamic language :)

By far the most frequently asked question was why dynamic languages? The advice that resonated the most with the folks that I talked to was about using it to help them build internal DSLs for their applications. This is the idea that you can use DSLs for part of your app, dynamic languages to help faciliate those DSLs, and statically typed languages for the foundational pieces. You'll find that this idea is often called polyglot programming.

The Talk

The IronRuby talk was the very last talk of the conference. Apparently I was one of the ‘big guns’ that would convince folks to stay all the way to the very end of the conference. While I prefer being at the start of the conference so that my talk can help drive further discussions with folks while they’re still there, it is what it is. Oh well, let’s see if the tubes can help start a conversation :).

My talk focused on three things. First up was an introduction to Ruby since half of the folks in the room hadn’t used Ruby before. I wrote a simple unit-testing framework live on stage. Each new feature in the framework helped to introduce a different feature of Ruby. At the very end of the talk I showed how that same framework could be used to test .NET code. Here it is in its entirety:

And here is some .NET code that we tested using this framework:

It worked rather well, and I think I’ll continue to use it at talks where I need to introduce Ruby to the audience.

Next, I talked about our Silverlight integration. I showed the excellent set of demos that Jimmy created for RailsConf last week, including his client-side Try IronRuby demo:

and the lovely Silverlight watch demo:

I also got a chance to sneak in some of the code I was hacking on in the evenings – an adapter that maps the HTML 5 Canvas API to Silverlight. I ran a few of the examples from the excellent Mozilla Canvas tutorials using a my adapter. I pasted some code (with minor rubification) from the Mozilla tutorial into the text box:

and hit run - the colored boxes with the translucent circles appeared below:

There's no reason why we couldn't do this with managed JavaScript either using the cross-language interop features of the DLR.

Oh yeah, there was one more thing that I demo'd - more on that tomorrow.

IronRuby and Rails

IronRuby dispatched some simple requests through an unmodified copy of Rails a few days ago. Today, we’re going to show off our progress live at RailsConf. This is an important milestone for IronRuby; it’s our ‘ticket to entry’ to the world of alternative Ruby implementations.

We started our work on IronRuby back in February 2007. Now, just 15 months later, we’ve reached what others are calling the “Rails Singularity”. A few folks claimed that we would never get here this quickly, or that we wouldn’t be allowed to accomplish this goal. But we did it on our own, in our own way and with help from our community. And we’re just getting started.

I have always maintained that you must judge us based on our actions and not our words. Running Rails shows that we are serious when we say that we are going to create a Ruby that runs real Ruby programs. And there isn’t any a more real Ruby program than Rails. This demonstrates that we’re true to the language, and that we’ve put compatibility above all else on our TODO lists.

But we have a lot more work to do.

Our performance is nowhere near where we expect it to be, particularly in startup of a large application like Rails. We are consuming much more memory than we would like to. But this is the price you pay when you put compatibility ahead of all other work. We’ve shown that we are willing to do what it takes to run Rails. Now we have to do the work to make it run better, and faster.

But there are other things to talk about as well.

IronRuby doesn’t just let you run Rails; it lets you interact with the rich set of libraries provided by .NET. You’ll be able to use IronRuby to build server-based applications that run on top of ASP.NET or ASP.NET MVC. You’ll be able to use IronRuby to build client applications that run on top of WPF or Silverlight. You’ll be able to use IronRuby to test, build and deploy your .NET applications. You’ll be able to run Ruby code in your web browser and have it talk to your Ruby code on your web server. That’s a feature that we feel that many folks will enjoy.

Perhaps even more important than all of this technical stuff is what the IronRuby project represents at Microsoft. IronRuby has pioneered a number of new processes that make it easier for other folks at the company to build and release Open Source products. What we learn from building IronRuby will be applied in other product groups to help us become more open and transparent than we have been in the past. We have a great leadership team that is willing to push the envelope on openness and transparency to create a world where both Microsoft and our customers can benefit.

Come join our project on Rubyforge and help us show everyone how it’s done!