Scipy 2010, Day 1 (posted 28 Jun 2010)

FIRST OF ALL:

People in the lower 48 don’t “get” coffee. Or, at least, lower 48ers that aren’t from Seattle. I haven’t been to Seattle though. Seriously, I’ve had a Hell of a time finding coffee so far, and it hasn’t even been that good. I ended up adding plain coffee and honey to my mocha for God’s sake! INCORRECT

Of course, jet lag makes coffee more important. I may have woken up 7:00am Austin time, but the 3-hr time zone difference means I was waking up at 4:00am AKDT. >_<

Update: The crap I ended up replacing my mocha with is so terrible it actually makes me angry.

Okay, so, Scipy.

I’m doing the advanced tutorials. Almost surprisingly, it/they seem to be a good fit so far.

Advanced Numpy:

I learned about a lot of little things, but I think the most immediately useful things to me are:

Fancy indexing: Use arrays to specify indices instead of slices.
Advanced broadcasting usage: Same idea as multiplying a vector by a scalar, but applicable to all m-degree and n-degree arrays.

For an example of broadcasting, check this out (from ipython):

In [158]: (i,j)=np.ogrid[0:3,0:3]

In [159]: i+j
Out[159]: 
array([[0, 1, 2],
       [1, 2, 3],
       [2, 3, 4]])

In [160]: (i,j)
Out[160]: 
(array([[0],
       [1],
       [2]]), array([[0, 1, 2]]))

ogrid works a lot like meshgrid (or mgrid for that matter), but instead of returning “full” structures, it just gives you a single “column” in the necessary direction, such that when used as in meshgrid, broadcasting works and you only really notice less memory usage and more fasters. Really slick.

Now, for an example of Fancy Indexing: We can get the diagonal values out of i+j like this:

In [161]: (i+j)[arange(3),arange(3)]
Out[161]: array([0, 2, 4])

Basically, the passed arrays are broadcasted with each other to make sure they’re the same size (in this case, they are), and then elements of (i+j) are extracted by-elements of the indexing arrays. Cool!

The other take-home lesson, I think, has to do with the clashing of paradigms between lists, iterables, etc. in the standard python world and the “vectorized” style of the scipy and MATLAB worlds. I mentioned earlier that I’d love to be able to extend list comprehension style to act naturally in the context of a 2-d array. In my mind, this would work something like this:

array(f(i,j) for (i,j) in iter_ij(A))

If we use ogrid and .reshape, we can do something like this:

array([f(i,j) for (i,j) in ogrid[0:3,0:3]]).reshape((3,3))

which is pretty crummy. Alternately, you can do basically what I did in my first example:

(i,j)=ogrid[0:3,0:3]
f(i,j)

where f(i,j) rolls vectorized style. I think this is probably the best you’re gonna get, and while it isn’t a list or a one-liner, I think it’s actually pretty clean. You could probably throw some sweetener on there too so you can say something more akin to f(i,j) for i,j=1,2,3, maybe doing something like:

def indices(xs):
    return (array(xs),array(xs)[:,newaxis])

But who knows?

This tutorial was given by Stefan van der Walt, who seems to be a pretty cool guy! I’m ~~stalking~~following him on github now.

Signal Processing in Python:

I learned about scipy.signal, which has a lot of slick shit! I’m not really a signals guy (though I learned enough from Controls for it to make sense), but the tools are totally there for being able to do Neat Things with signals. Also worth noting is that pylab has a few controls-specific plots.

In fact, the instructor of the tutorial is working on some extras to make it even better., by including implementations of MATLAB’s nicer tools. Definitely worth checking out!

Lunch:

I’m just going to take this opportunity to say: The Austin way of saying “Guadalupe” (“Guad-a-loop”) is really grating. I swear these people are tollin’. Also: What is up with this intersection? It looks like a two-year-old designed it. Shame on you, Texas. Shaaaaaame. Oh, and shame on you for your lack of coffee.

Other than that: I feel like I’m being a bit antisocial so far. This guy from Norway introduced himself and suggested that we “team up” for lunch-getting. I had to go pee, and I kinda ended up ditching him on accident. I feel bad!

Oh well.

HPC:

This is a pretty cool tutorial! I’m not really typing along as much because I don’t have some of the tools (just now installed Cython) and, well, my laptop has an Atom processor. I’d just feel silly trying to parallelize code here. There’s also the unfortunate bit that some of the techniques rely on the Enthought distribution, which is a cool thing but wraps in some proprietary things I don’t really feel like dealing with.

Still, I’m seeing a lot of cool examples, and I learned a bit about the GIL. ~~I'll have to show off my "Escaping the GIL" diagram, even if it doesn't mean much.~~ Here’s my diagram showing how to avoid the GIL by writing your parallel functions in C(++):

Run!

An idea this gives me is to try and use haskell to write foreign functions which we may call from python, as detailed here, based on this technique. What could also be good is possibly using such tools as cython to mediate the process. I’m not actually sure if that’s possible! But wouldn’t it be bitchin’ if it were? Yes it would.

Alternately: Could it be worth doing distributed browser-side javascript? >:)

This tutorial was taught by Brian Granger. *follow*

ipython -p sh:

Holy crap! This is awesome! I had no idea. Check out this screenshot:

!!!

(flickr)

I bet this would be particularly nice with Windows, whose standard cmd honestly kinda sucks.