How much time should be spent on coding?

Probably any scientist that went through college starting sometime in the early 2000's knows how to program to some degree. They may have learned how to write computer code to do data analysis, to implement simulations, or simply for fun. In my own case, I had been interested in computer programming since middle school when I first taught myself Visual Basic and HTML (I know, I know, HTML is not a programming language, but it did get me interested in programming).

Now, as a researcher, I often have to temper my enthusiasm for writing programs. It's true that creating scripts to automate data analysis and processing will save me a lot of time in the long run, but I also think that tweaking my code too much or trying to engineer it to do too many things may be counter-productive. After all, code is a means to an end, and my goal is not to write as many programs as possible.

So what's the right amount of time that should be invested into writing code to help with usual laboratory tasks? This is a tough but important question that scientists should ask themselves, since it putting in just the right amount of time should maximize their productivity.

And beyond maximizing productivity, scientists should also dedicate time to writing code that makes it easier to document and reproduce what they did. For example, I have recently written scripts to take the output of a curve fit routine and write a report that includes the fitting bounds, initial guesses, fit functions, and other data that is relevant to reproducing a fit. Hopefully, a future grad student can read my automatically-generated reports and figure out exactly what was done.

So in the end, enough time should be invested in coding when it significantly cuts down on the amount of time taken to do repetitive tasks and when it streamlines documentation and note taking, which may not happen otherwise.

How do you teach what polarization is?

Today is Optics Day at CREOL, our annual public open house where we present demonstrations of various optical phenomena and technologies, speakers, and pizza. :)

During this year's Optics Day I am charged with explaining the phenomenon of polarization to visitors. Now, I find polarization incredibly difficult to explain to non-scientists, and here's why: the usual treatment of optical polarization in physics involves describing the direction of the electric field vector of an electromagnetic wave. If I were to start with this definition while speaking with somebody not trained in physics, I would then have to explain electromagnetic waves. This would be followed by an explanation of the equivalence of light and electromagnetic waves, wave phenomena in general, linear, circular and the more general elliptical polarization states, etc. etc. until the poor person who has come to see a cool demonstration and learn something new has completely been befuddled because it takes so much background understanding to comprehend what polarization even means.

This year, I am determined to find an explanation of polarization that is more intuitive to a non-scientist. A rough outline that I intend to give for polarization's foundation in observation goes as follows:

1) Our sense of sight is perhaps the most obvious sense we have. We see objects and from these objects we discern shape, size, color and other properties.

2) There are physical quantities that cannot be sensed by our eyes. For example, flowers have fragrance that our noses can detect. Wind is another example. We feel its effects or we see its effects on other things, but we don't directly see "wind." Therefore, there are physical quantities that cannot be seen but nevertheless may be sensed.

3) There are still more phenomena that exist but cannot be sensed by any of our sense organs. For example, a compass points north because the needle experiences a magnetic force. Additionally, small objects all fall towards the earth because of gravity. Magnetism and gravity require tools that sense things that we cannot: magnetic and gravitational fields. Where our senses fail us, we use tools to measure some quantity.

4) Polarization lies in this last classification of phenomena. It cannot be sensed by us (which isn't strictly true), but can be determined by appropriate tools. These tools are things that are found in nature, like quartz crystals, and man-made objects like polarizers and waveplates.

From this foundation, I will explain some of the consequences of the polarization of light, what it can be used for, and may even digress into the physicist's model if the visitors are interested enough. My hope is to build the concept of polarization up from a basis of observation, not to start with our model first, followed later by how we observe polarization.

Increase a Python program's memory efficiency with generators

I've recently been working with very large data sets (more than a million data points) and have encountered a serious reduction in the efficiency of my Python programs' computations. One reason for this is that I have been reading a large data file into memory all at once, performing my computation on this data file, and then moving onto the next file, all the while keeping my calculation results in memory so that I can plot them at the end.

One tool I've discovered for increasing the efficiency of these types of operations is the idea of a generator. Now, I realize that these are well-known in Python circles. However, I am not primarily a programmer, so I am not fully aware of the tools available to me when I write programs. Hence, I sometimes use this blog as a notebook so I can easily find these tools again in the future.

Simply put, a generator is an object that executes commands and yet may be iterated over like the elements in the list. One example, shown here, is the readline() method. The code looks like this:

inFile = open('sampledata.txt')
w = inFile.readline()

w is a generator function that reads only one line in the file into memory. Each time readline() is called, w is assigned the string representing the next line in the file sampledata.txt. According to the above link, an object that offers the abilities to load just parts of an object into memory and yet still iterate over all the parts is a generator.

Another nice description of the use of generators is given here. And, because I am having some trouble plotting all this data, a discussion of possible solutions to plotting large data sets is presented here at Stack Overflow.

A primary school method for multiplication of numbers

I found this link that was posted by a friend on Facebook this morning. It's a pictorial description of an algorithm for computing the multiplication of two-digit numbers. I haven't had time yet to think about it, but so far as I can see now it's useful only for computing the product of numbers with one or two digits. Still, it's pretty cool.

In 2011 I had a calendar in my office on mental math algorithms called Lightning Calculation, which I believe I also mentioned in an earlier post. Check this out if you wish to train your brain in the powers of mental computation!

Important equations and numbers to memorize

Many students in science and engineering today claim that the need to memorize certain numbers or equations is outdated with the advent of the internet and the ease with which we may find information. As an undergraduate, I often heard my engineering colleagues complain when they were not offered an equation sheet for a test because real-world engineers don't have to look up "well-known" equations when solving problems.

However, I contend that there are advantages to memorizing a few important equations and numbers. These reasons include the following:

1. You may work faster since you will not need to constantly reference those equations.
2. You may mentally check your work to ensure that the numbers are correct.
3. You may mentally approximate the solutions to certain calculations if you don't have access to reference material. I remember reading once how Feynman would challenge mathematicians by approximating powers of numbers in his head by knowing the natural logarithms of 2 and 10.
4. You may impress the ladies with your abilities (not guaranteed).

So what are some equations and constants I've found useful throughout the years? Here is my list:

1. The binomial expansion
2. The Taylor expansion
3. The power series representation of e 
4. The speed of light in vacuum and the impedance of free space (about 377 Ohms)
5. The refractive index of common glass (~1.5) and water (~1.33)
6. Boltzmann's constant and the thermal energy at 300 K (4.14 Joules) and thermal voltage at 300 K 0.026 eV)

Other important things to memorize, depending on your field, include:

1. Stirling's approximation
2. The Krebs cycle (a biologist once told me that many ideas in molecular biology come back to this)

Consider these as tools in your toolbox. They can make your job a lot easier.

Addendum: Not memorizing the basics is a mistake to avoid in physics according to this Back Page of APS News.

Working around distractions

I've been trying a method for enhancing my productivity during the day. Why am I doing this? Well, I want to maintain a healthy work-life balance; if I get more done at work, I have more time to spend with my fiancee, workout, cook, read, etc. This is incredibly difficult in academia, where there is no limit to the work that can be done. However, I value my personal life as well as my professional one, so a balance must be found.

My approach is simple and is inspired by the No Meat Athlete and Zen Habits blogs. I first make a list of things that need to be done. It can be large or small; it doesn't matter. I then work on one thing on the list and only open programs and browser tabs that directly relate to this task. That's the most important part. Multitasking, I've decided, is the bane of productivity.

If I see an interesting paper, I use Instapaper to save it for later. If I have too many e-mails or unread items, I go through and delete the ones that were once relevant but no longer are. The goal is to reduce the noise and possibilities to be distracted while working. This also includes choosing not to listen to music if I am writing anything or doing something that requires more than passive attention.

Of course, I also account for when I can not avoid being distracted. I allow time to be interrupted (and I will be numerous times) during the day by my colleagues and adviser. I also set time limits that are proportional to the magnitude of the task. If I have lab work, I give myself an hour minimum and two hour maximum; conversely, computer work is usually delegated 15 minutes to an hour. By setting upper time limits, I allow myself to be satisfied with unfinished work, knowing that when I pick it up again I will have a fresher mind. And by setting lower time limits, I ensure that I have enough time to accomplish something meaningful. Once an item on the list is done, I remove it. If it's not done, it stays.

This may be a bit too much micromanaging for some, but with the amount of things that are required of one in academia, I think it's necessary.

I believe that Henri Poincaré had habits similar to these. His goal, in part, was to maximize his creativity. I think this was so because he believed that a creative mind must be allowed to wander around different tasks. In this way, he recruited his subconscious to work on problems while he consciously worked on others.

A resource for volume visualization in Matlab

As many engineers may already know, visualizing scalar or vector volume information is difficult. Examples include the heat distribution in a room (scalar) or the electromagnetic field around a complex-shaped scatterer (vector). There is a good set of resources on how to do so in Matlab here: http://blogs.mathworks.com/videos/2009/10/30/basics-volume-visualization-29-examples-of-scalar-and-vector-fields/. I haven't tried volume visualization with any Python packages (see Python x,y for a collection of useful scientific packages), but if I find the time to play with it, I'll compare and contrast the two.

A focus on statistics

My biology colleagues know a lot about statistics. They routinely perform hypothesis tests, ANOVA, and perform Box-Cox analyses to be sure that the residuals of a fit to data are normally distributed. Ask many physicists and chemists about these tools and you'll likely be met with a blank stare. It could be argued that this points to a problem in physics and chemistry education. To some extent I agree that we (physicists) are not well trained in the art of proper data analysis.

On the other hand (and based on my limited knowledge of the biological sciences), I wonder if biologists place too much emphasis on statistics. If a student's first thoughts in a data analysis are about which type of regression is valid for the data, then I fear that they may miss obvious trends that may answer that question for them.

My belief is that data analysis is best approached intuitively first and formally second. I'm also afraid that a biology curriculum that focuses on the technicalities of statistics may under-emphasize this "human aspect" of analysis. Likewise, there is a point where rigor must be included and I see many physical scientists unable to provide it.

Again, these thoughts are based on my own limited understanding of the biological sciences. There are biologists, physicists, and chemists who excel in all areas. But differences in curricula and courses may bias us towards one aspect or another when in reality good scientists are capable of both.

Being rational with science

There's a cool talk posted at the blog Measure of Doubt that was given recently by one of the blog's authors, Julia Galef. The talk concerns the idea of a straw Vulcan, an idealized character based on Star Trek's race of ultra-logical humanoids. Galef argues that the Vulcans base their actions and decisions on a logic that's popularly perceived as rational, when it is in fact not. This is because she defines rationality in one of two related ways: 1) a method for obtaining an accurate view of reality, and 2) a method of achieving one's goals. To make her argument, she presents five beliefs about Vulcan behavior that are commonly held to be rational and then gives examples from both Star Trek and real life where this behavior has violated her definition of rationality.

I particularly like the second and third items on her list—never making a decision on incomplete information and never relying on intuition—because I find that these are common mistakes that scientists make. For example, suppose some graduate student wishes to setup an experiment that he or she is unsure will work. The student may take one of two courses of action (really there are three, the third being a combination of the first two). The first is to try the experiment and see if the outcome is desirable. The second is to carry out a number of calculations to determine if the desired outcome will be produced, and then perform the experiment. The fallacy occurs when the student attempts to plan too much and wastes time on arduous calculations when the experiment may have consumed less time. This is a case of failing to act simply because he or she did not possess the complete knowledge of whether the experiment would work in the first place.

The example above is irrational by Galef's definition because, in all likelihood, the graduate student would have liked to have obtained a yes-or-no answer to the question "does the experiment work?" in as little time as possible, and sometimes this means running the experiment before fully understanding what the outcome would be. Of course, it takes intuition to determine when it's time to put down the pen and paper and do the actual lab work, and that's why it's rational to rely on intuition.

In a sense, these arguments depend strongly on Galef's definition of rationality, but I see no reason why this isn't a good definition to work with.

Deconvolution—I want it all!

A groupmate and I were discussing practical deconvolution of an instrument's response from data the other day. After some searching on the internet, he came across the following words of advice:

When it comes to deconvolution, don't be greedy.

This is actually pretty good advice. In theory, the idea works great. Fourier transform the data, divide by the instrument's transfer function, then inverse Fourier transform to get the deconvolved data. But with real data, you risk amplifying the noise or introducing artifacts, especially with FFT-based methods. This last point is pertinent if your instrument's impulse response is very short compared with the span of the data. So use some caution; just because you can deconvolve, that doesn't mean you should.