PYTHON VECTORIZE FOR LOOP CODE

Because I think any serious R programmer should know about and use these functions whenever possible, I was surprised to see this around the Twitterverse the other day:Ĭompute row sums of a matrix, M, with apply(M, 1, sum) #rstats - One R Tip a Day January 28, 2014

There are two ways in which matrices sometimes act like vectors: way 1: rows (or columns) are the vector entriesĬonceptualizing a matrix as a one-dimensional array of rows (or columns) lets us add a bunch of vectorized matrix functions to our quiver! The following vectorized matrix functions exist and are super fast and awesome: rowSums, colSums, rowMeans, colMeans (in base), rowSds, colSds, rowVars, colVars, rowttests, rowFtests (in genefilter), colMedians, rowMedians (in matrixStats). I’ve learned this lesson pretty recently. Can I somehow incorporate this into my loop code? lesson 2: matrices sometimes behave like vectors Am I calling a vectorized function inside that loop? Most functions that take a number or a string as input are vectorized. This is what I’ve been doing recently: every time I write a loop or apply statement 3, I think about whether I can do better. lesson 1: think twice about every loop or apply statement you write If you are working with data frames and find yourself using lots of apply statements, check out plyr, which provides some really nice syntax, and dplyr, a superfast “next iteration” of plyr. (I did get a 2x speedup by using vapply and telling it that I knew the result would be numeric, but that’s nothing compared to my 100x speedup from using vectorized log.) > system.time ( sapply ( nums, log )) user system elapsed 1.847 0.025 1.871 > system.time ( vapply ( nums, log, 0 ) user system elapsed 0.781 0.001 0.781 As the R Inferno puts it: apply “is not vectorization, it is loop-hiding.” So using apply instead of loops might make for nicer/shorter code, but it won’t make for faster code. The point I want to emphasize is this: apply functions are basically equivalent to loops in terms of speed. The apply family of functions provides some fairly clean syntax for apply-ing a function (any function) to each element of some data structure. If, on the other hand, you are acting as an R developer, you might have to implement your own vectorized function, which will probably involve writing some C or Fortan.

PYTHON VECTORIZE FOR LOOP CODE

But if you’d like to write R code that will run in a reasonable amount of time on large dataset (memory issues aside), it’s good to think about USING vectorized functions, which consequently means learning where and when to look for them.

If you are acting as an R user, you don’t need to worry about WRITING vectorized functions. So in conclusion: vectorization is important because it allows you to operate on vectors quickly (unlike looping). time ( log_novec ( nums )) user system elapsed 1.938 0.004 1.941 > system. > log_novec = function ( n ) > # timing results: > nums = sample ( 1 : 10, size = 1000000, replace = TRUE ) > # my function to call log on each vector element separately:

# illustrating log's behavior: single values and vectors

Python vectorize for loop