Welcome to the June 2015 Practical Skills for Scientific Computing Workshop
This is an interactive notepad for group note taking and interaction. Feel free to make your own notes here and add to those of others.
We will also put code snippets here and get you to contribute answers to challenges we set
Useful links:
- The Scientific Computing Facility wiki:
- The course organization site
- Software Carpentry Resources
Link for materials:
https://indico.mpi-cbg.de/event/6/other-view?view=standard
Text Editors:
Mac OSX: TextWrangler (http://www.barebones.com/products/textwrangler/) is Great
Windows: Use Notepad++ (http://notepad-plus-plus.org/)
Linux: Emacs, Vim, gedit, nano ...
all of the above are free to download and use (just google and download)
For those reading here I have now enabled the course evaluation at:
https://indico.mpi-cbg.de/event/6/evaluation/evaluate
Please fill it in before your leave today :)
ONCE YOU ARE SETUP WRITE YOUR NAME BELOW:
Peter Steinbach
Max Krause
Stephan Janosch
Misho Sarov
Marija Matejcic
Helena Jambor
Britta Schroth-Diez
Akanksha Jain
Monika Okrouhlikova
Christopher Schmied
Yu-Wen Hsieh
Ivana Henry
Nikolaj Zuleger
Ian Henry
Guendalina Marini
Anna Klemm
Johannes Schindelin (╯‵□′)╯︵┻━┻
Antje
CHANGE VIEW of the shell: right click, show inspector
INTRODUCTION TO SHELL AND COMMANDS
whoami > prints user name
prompt, standard output
Britta: unix cheat sheet .pdf check in google
pwd - present working directory / print working directory :-)
/ - root directory in your file system
commands have arguments, separated by spaces
ls - list current directory
ls -ltra shows the directory content time sorted reverse
-l - long
-r - reverse
-t - sort by time
-a - all
--color - why not have some color in your life? Does not work on Mac OS.
cd - change directory (with relative path -> goes down one level, with .. -> goes up one level)
-F adds a slash to you list output
can go back to already used commands: up or down arrows
[TAB] autocompletes file names
Tab key fills out a started name (like "June15_17_material" if you only start Ju)
no slash: relative path
with slash: absolute path (copy output of pwd, add the rest)
do not use spaces or dollar signs or minus or other things that have a special meaning in Unix in file or directory names!, also https://wiki.mpi-cbg.de/compdoc/Filenames
everything that starts with a dot is hidden by convention (-a shows it)
double-dots -> to go up one level with any command
one dot -> works in current directory level
flags are cool! (flags or arguments?)
ls ./folder name
Some art: http://patorjk.com/arial-ascii-art/#image=poohbear
,-·-,· ~ ,
, - ' ',' ,'- ,
, ' ' , ' ' - ,
; ~ ',
', 0 '',' '#,
,- -,' ;-'
,'· ' - , )., . - '
Ñ맧 ,' , - ~ ' , , -'
, - '-,' ' ~ - '-, - ~ ~ ,
; , . , / ,'' ) ,'
,'·' ' -, , , ' ,' , -'
, ' ; ;, ' ; . - '
,' , ;- , ;
,' , ; ' - , , , , ,'
, - ,' ,'
· ' - . - ' -
' ,, ~ ' ' ~ , , '
', ' , , - ;
', ;¸ , . - ' ,'
', ,' ; ,'
', ,' ,' ,'.-~ -,
; ;-~'- , ,'
,' ', , , , , . -'
', ,., , , , , . . 'ƨ‹
CUUUUTE
Terminal Session, June 15, 3pm:
(green)
- finally know what ~ means
- learned about terms seen on internet/papers
- etherpad, +2
- like new commands
- tab completion
- history with arrow keys
- repeat tasks to understand better
- well explained, +2
- speed is good, +1
- starting with basics
- fighting fear of CLI
- navigation
- like sticky notes
- Ian+Stephan to help
- ../.
- ls
(red)
- confused where I am (cd vs ls)
- list of commands, +1
- already know many of the commands
- my terminal does not have all the commands
- don't use etherpad as I am busy typing
- more time to try examples
- confusion on if to follow suite and if not
ls --help > works only in linux not mac
man ls >> gets the whole list for commands, flags etc
man cd >> gets manual for the specific command
to search in manual: /SEARCHTERM
navigate arrow up down
q to quit man pages
mkdir {name} > make empty directory
touch {name} > makes new file or updates timestamp of existing file
command line editor: nano
nano comes with every terminal installation see other editors above
nano {name} > enters or creates file
below see the options of nano (navigation, saving etc)
exit with CTRL + X, answer Y
rm {name} > remove file
Be careful there is not bin for deleted files
rmdir {name} > for removing a directory only if empty
rm -rf {name} > removes the directory and contents -r recursive -f false for prompt remove
rename files
mv {name source} {name target} >> moves files when giving path but also used for renaming
cp {name source} {name target} >> copy paste files
-r flag copy entire directory
last argument is the destination
first is source
wc - word count (output: line count, character, word counts)
wc -l - count lines
* for wildcard expressions
example: wc *.pdb > matches everything that contains the suffix .pdb
wildcard expression can be evaluated to give commands to multiple files
other wildcards:
? - is a wildcard for only one character
ctrl c to cancel any operation
save entire output to new file
standard output redirection: >
sort - sorts output
-n by numbers
-r reverse
head - shows the head of the file
tail - show end of the file
PIPELINES
pipes and filters paradigm
| pipes stdout of one program into the stdin(put) of another
used to daisy chain small programs together
http://www.lovesdata.com/wordpress/wp-content/uploads/2014/02/regex-cheat-sheet.gif
use | to pipeline commands , pipes and filters (unix)
redirection to a file is a side note in the paradigm of pipes and filters
standard input redirection: <
wildcard expression:
[AB].txt > matches either A or B
$ indicates that it's a name of a variable
for filename in "basilisk.dat unicorn.dat"
> do
> head -3 $filename
> done
is the same as
for filename in "basilisk.dat unicorn.dat" ;do head -3 $filename ;do
touch 'red dragon.txt'
rm red\ dragon.txt
escaping work by putting a \ before a e.g. space character in the shell
echo - writes arguments to the stdout
echo before commands shows you what the command will do without actually doing it -> safe way to try things first
output redirection via > will overwrite content of the destination file, via >> will concatenate to the content
Play around the the commands:
http://web.mit.edu/mprat/Public/web/Terminus/Web/main.html
hint: there is no cat command, use 'less' to look at things Great page!
an advanded command line game
http://overthewire.org/wargames/bandit
https://github.com/jlevy/the-art-of-command-line
Final reward for the day ONE:
, - ' ' ' ' -,
, ' ,' , , , , ¸
,' , -' , - ' , - - - , ' -
,' , - ', , - - ~ , ,' , -' ,'.-'
; , ' '-,',' ' .' , -~'
, ·~; ~ ' , , '
, - ' ', ~ ;,'
, ' ', , - ©, ,'
; ', , -', , ,'
, -' ' - - ' ' - .. ', ~'-, ¸¸¸¸¸
, ', . , ' ,'-' ' - , , , , , ,)
' , ' , -, ' ,
' ' ' ' - , ', ', ' ,
Ñ맧 , - ' - - ~' ' - -~ ' ' ' ' ' - ,
Shell session, June 15, 5pm:
(red)
- loops are a bit difficult
- cat $filename?
- slow on the easy, speed-up on complex stuff
(green)
- nice explanation
- good immediate help
- more confident
- cool course
- easy to follow
- learnt for loops
- learning curve
GOOD MORNING
, .. , . ····· . , .. ,
' , ''~ ~ '' , '
,' \ / ',
; @„..„@ ;
', \/ ,'
,·' ', '' ~~'' ,' '·,
,' (( , ' ~~~ ', )) ',
; ((( ,' ~~~~ ; ))) ;
', (( ', ~~~~~ ,' )) ,'
' \)\)'·, ,·' (/(/ '
\ '| |''| | /
==='()()()()()()'===
\ \ / /
\ \ / / Ñ맧
\ ''' /
''··''«•š
CLI is art!
variables are alive as long as shell session is alive
use curly brackets for proper calling of a variable: ${variable}
content in square brackets is always OR
goign back multiple levels:
../../../..
wildcard expression:
[AB] = A or B
[a-zA-Z] = a to z OR A to Z
ls -l: -l flag gives you more info, first permissions, user, size (in bytes), date and time of last access
count number of file:
ls | wc -l
for dataset in *[AB].txt; do echo ${dataset} stats-${dataset}; done WHY? CHRis, was that you? see line 277
shell script 1:
# defines variable "dataset" and starts the for loop
for dataset in *[AB].txt
do
# writes source file and the output file into command line prompt
echo ${dataset} stats-${dataset}
# calls shell script "goostats" with input file and tells it how the output file needs to be named
bash goostats ${dataset} stats-${dataset}
# ends for loop
done So for every program we want to run, we use => "program" "input" "output"? yes usually you have a command and then define input output. But this depends on how the command is defined
for number in 1 2 3 4 5 6 7 8 9 10; do echo ${number}; done
alt writing:
for number in `seq 1 10`; do echo ${number}; done
for zero padding: "%02d"
how to write your own shell script:
aim >> print the middle of the file
open a new file with nano
nano middle.sh
convention: shell skripts should end with .sh
head -15 octane.pdb | tail -5
in nano for saving without closing: crtl + o
execute middle.sh:
bash middle.sh
head -15 "${1}" | tail -5
execute:
bash middle.sh {file}
${1} in shell script will be defined as the first input
example1: print_args.sh
echo "${1}" "${2}" "${3}"
execute in prompt:
bash print_args.sh {file1} {file2} {file3}
example2: print_args.sh
echo "arg one" "${1}"
echo "arg two" "${2}"
echo "arg three" "${3}"
execute in prompt:
bash print_args.sh {file1} {file2} {file3}
First trial~
tomancak-mac-21-wifi:molecules hsieh$ bash middle.sh SWen boy
SWen is
a very handsome boy
tomancak-mac-21-wifi:molecules hsieh$ cat middle.sh
echo ${1} "is"
echo "a very handsome" ${2}
WHAT IF
a nice book with a title called "what if"
https://whatif.xkcd.com/book/
adding comments: using # after this text will be ignored until the next line starts
shell script sorted.sh
# count the number of lines of all files given as arguments
# print them out as a sorted list
# Usage: bash sorted.sh <file1> <file2> ...
wc -l "${@}" | sort -n
execute:
bash sorted.sh p*.pdb
bash sorted.sh p*.pdb octane.pdb cubane.pdb
"${@}" wildcard for every file (argument to the script) given afterwards (after the script name)
do-stats.sh
# Calculate reduced statistics for data files at J = 100 c/bp
for datasets in "${@}"
do
echo ${datasets} stats-${datasets}
bash goostats -J 100 -r ${datasets} stats-${datasets}
done
execute:
bash do-stats.sh *[AB].txt
bash do-stats.sh *[AB].txt | wc -l
how to find stuff: using grep
# shows entire line were the input string is present
# grep will match any string
grep --color not haiku.txt # flag --color for color highlighting
# to search for a specific word use the flag -w
grep --color -w day haiku.txt
-n # flag shows the number of the line where it found the string
grep is case sensitive!
-i # ignores case
-v # do not show me these lines
combine all the flags:
-niv
grep "it?" data/*
Regular expression, powerful tool to match patterns in a string:
grep -E '^.o' haiku.txt # ^ is first Letter in a line then search for everything were the second letter is a o
Using FIND
# find within the current directory the type directory
find . -type d
# find within the current directory the type file
find . -type f
-maxdepth 1
-mindepth 1
# search for a file with the suffix .txt
find . -type f | grep .txt
or
find . -maxdepth 1 -type f -name "*.txt"
# counts the number of lines with in the file with txt
wc -l $(find . -name "*txt")
or
wc -l `find . -name "*txt"`
IMPORTANT: needs ascii text file does not work with binaries (excel, words)
LUNCH BREAK
____
¨¨¨¨¨¨\\
_______ ¸, ----•¬ -- ¸ \\¦<)
\_____ \ (_________,' \ \
>===,\ \______, /'__ \ # /°\ # / ¦¦ /' \\______
, \ \ \ \ | / ¦|___| ##(®)## || ;;/'\\,;-----/.,
( ,;' ---¯¯¯¯¯¯' /,' ====== '' / '// ,;' / /,.\ \ ,',\
( (________)) ___/ / < __ > O//// / /( ( © ))))
(____________,/ =======" 0 " // \ \ ' _ /"/
"- , ___ , - " ¯¯¯¯¯¯¯¯¯¯¯¯¯¯ " - ,___-/§Ãž
June 16, shell, 12.30pm:
(red)
- bit too fast
- wait for everyone to arrive in current dir
- $ still hard
- more challenges
- don't move between too many folders
- a lot of info to remember
- cannot use high comma?
(green)
- good stuff, +5
- really interesting
- game links in etherpad
- learned scripting
- combinations good
- great tools
- variable name best practises?
- terminal short cuts
python
needs the python interpreter to execute the program imediatly
call python interpreter with:
python
then a prompt appears. To leave python interpreter in the terminal with:
crtl +d
start python interpreter with
ipython notebook
new
-python 2
opens interactive source code environment
# import python library
# a programming language like python comes with certain capabilities
# other not included capabilities need to be imported with libraries that
# extend the functionality of the language
import numpy
creates array, gives function like mean, max, min etc.
shift + enter
executes the line
# function loadtxt with options fname (file name) using a delimiter
numpy.loadtxt( fname='inflammation-01.csv' , delimiter = ',' )
# loads the array into memory when executed
text highlighting in ipython editor
this can also be used in the terminal
variables: name = value
weight_kg = 55
# print variable: use print function and then the variable
print weight_kg
# a variable can be a number, string object etc
# this puts this information away for later use
mass = 47.5
age = 122
mass = mass * 2.0
age = age - 20
print 'mass in kg =' , mass
print 'age in years = ' , age
# in interpreter typing variable name only gives value
weight_kg = 57.5
print 'weight in pounds: ' , weight_kg*2.2
# for string use high commas:
'' single high comma
"" double high comma
both comma types are interchangeable
# asign array to new varible
data = numpy.loadtxt( fname='inflammation-01.csv' , delimiter = ',' )
print data
# print type of varible
print type( data )
# prints out the dimensions of data, in this case rows and columns
print data.shape #size(x)
print dir(data)
# Information about data type
help(data)
numpy.ndarray online docs http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.html
IMPORTANT: python (like c, java) starts to count at 0
# access first row and first column of this matrix
# reference individual items in matrix
print 'first value in data: ' , data[0,0] #fuses int and variable (can be Int or Number)
# Slicing an array
# access a row or column
print data[0:3,36:40] # start with row 0 end with row 2, start at column 36 end with column 39
print data[:3,36:]
right open interval means that always the right (biggest) number is excluded from interval[:3,36:[
print data[5:10, 0:10] # start with row 5 end with row 9, start with column 0 end with column 9
# strings are a collection of characters
my_string = "hello world"
print my_string
# we can also index a string
print my_string[0],my_string[1]
# and also slice the string
print my_string[:3]
# start with - would start to count from the end
e.g. data[-1]
# data operations: element wise in a numpy ndarray
doubledata = data*2.0 # multiplication
tripledata = doubledata + data # addition
print doubledata
print tripledata
# use function mean part of numpy library
print data.mean() #performs calculation of the mean <> giving back the size in .shape; () as input for function <> [ ] indexing into an array
print 'avg inflammation: ' , data.mean()
print 'max inflammation: ' , data.max()
print 'min inflammation: ' , data.min()
print 'std inflammation: ' , data.std()
# start selecting patients and calling functions on them
patient_0 = data [0,] #all columns in row 0
print "max of patient 0: " , patient_0.max()
# operations accross axes row and column wise
# axis label in numpy library
# column axis 0: goes from top to bottom
# row is axis 1: goes from left to right
our_means = data.mean(axis=1) # mean accross each row
print our_means.shape
print our_means
print data.mean(axis=0).shape, data.mean(axis=0)[0]
print data.mean(axis=0)[0] #first element of the just calculated row of means
# for creating plots within the browser window of ipython
# specific to ipython
%matplotlib inline
# importing pyplot function form matplotlib library
from matplotlib import pyplot
# matplotlib = package
# pyplot = library
also: import matplotlib.pyplot
pyplot.imshow(data)
pyplot.show()
avg_inflammation = data.mean(axis=0)
pyplot.plot(avg_inflammation)
pyplot.show()
8
Did you get the
/¯¯¯\ ,·´¯¯'| |¯¯¯¯`·. /¯¯¯¯\
/ \ .·´ '/| |\ `·. / \
/ \.·´ / '| ____ | \ `·' \ ___
/ / /.·´ _ `·. \ \·. '/\ \ .·´ \
/ / / /_| '| \ \ `·. / |\ \ / .·-. \
/ '/\ .·´/ / / ______/|, '| |·. `;' / \'\ \ | | | '|
/ / /'´ / / | ¯¯¯¯¯¯/| \/ /| `·. | / \| |'|\ \.·´ /|
.·´ / / |.·´ / |\ ________/ '|/____/ | `' '/____/|| \____.·´ |
|¯¯¯¯¯¯| '/ '|¯¯¯¯¯¯| /| | | '/| | / | |'| \ | | '/
|______|/ |______|/ \'|_______ |/ '|____|/ -•X99•- '|____|/ \|___ |.·´«•š
??
element = 'lead'
print element[0]
print element[1]
print element[2]
print element[3]
my_string = 'aeiou'
for index in [0,1,2,3,4]: # for index=[1,2,3,4,5] ...end
# needs a indentation for hierarchy
print my_string[index]
print "done!"
alt:
# python will see that my_string is a container that contains a number of items
# the for loop will go through this container
for char in my_string:
print char
alt:
for index in range(4):
print my_string [index]
length = 0
for char in my_string:
length = length + 1
print "I found ", length, " characters"
# packs above now in a function that does the same job
print len(my_string), "should have been the answer"
# prints last character since it uses the last used character of the loop
print charhttps://mpi-cbg.de.etherpad.mozilla.org/6?
strings are immutable
lists are mutable
my_list = [1,2,3,4]
print my_list
print my_list[0]
print my_list[:2]
print my_list[-1]
# changes item
my_list[0] = 42
for num in my_list:
print num
print my_list
# adds new item
my_list.append(5)
print my_list
# turns the list around
my_list.reverse()
# delets item
del my_list[0]
# glob library contains only one function: glob
import glob
print glob.glob("inflammation*.csv") # allows to perform wildcard functions
END OF DAY 2
¸.,..,...,...,...,...,...,.¸
(¯\/¯\/¯\/¯\/¯\/¯\/¯\/¯)
\ `·¸ `·¸ `·¸ /
\`·¸ `·¸ `·¸ `·¸ /
\ `·¸ `·¸ `·¸ /
``-.\ `·¸ `·¸ / .-´´
\ `·._`·¸ _`·´ /
`==, ,==´
/ \
| ¦ |
| ¦ |
| ¦ |
| ¦ |
| ¦ |
¸ .—.,¸ | ¦ | ¸,.—.¸
(¸--------}-| ¦ |-{--------¸)
`·-—-´ | ¦ | `·—-·´
| ¦ |
| ¦ |
| ¦ |
| ¦ |
| ¦ |
| ¦ |
| ¦ |
/ ¦ \
¸-·´ ¦ `·-¸ 2LiVE4ƨ
June 16, python, 5pm:
(red)
- functions (power, faculty)?
- too bad in fast typing
- better chairs please
- explain data structure?
- confusing: different language (shell vs python), +1
- not intuitive, more practice
- biological background of dataset questionable, +1
- messy big chunks of code
- csv required for python?
- strange: loop with correct syntax
- slow in some phases
(green)
- know how to open ipython
- course is great, exactly what I need
- great + useful, +5
- faster
- good python intro
- ipython error messages
- ipython persistency
- programming doesn't look as scary now
- similar to matlab
- games are fun
- speed is good
GOOD MORNING
¸ ....¸
¸ · '' '' · ¸
¸'' '' · ¸
¸'' ''·¸
; ¸ ¸·'' ¸·'' ¸ ''¸
''¸ ¸· ¸'' ¸ ·''¸ ·''¸-~''¯¯¯'' · ¸
''¸ ; ¸'' '' '' ¸''¯¯'' ~·¸ ¸¸)
;\ '' ¸ ¸ '' ·¸¸ '' ·''¸¯'' ·.
„·''; ''„ '') ¸·'' ·¸ ''·¸'' · ¸'') ',
„· '' „ ; ''„ „'' ''·¸¸'' '' ',
„''¯'' ·''·~''¯ ,·' „'' „·'' ,·´¯`·. ;
; | ,' „ '' '' ,·´¯`·. ; @'; ;
''·„_'|.¸ ;'' ; @'; ' ··'' ;
'; '; ' ··'' ¸ · '''' ' ·,
''·„ ,·'' ` ·. '' (¯¯) ' ,
''('' ''~·.¸ , ·'.¸¸¸__¸¸.·''·· ;
`·...¸·~ `····´ , '
'·, ´¯ , · '
(''¯` · · ··¸·---........-----,·· ´~¸
''·¸¸ '' ··..¸¸____¸¸|¸) )
„.·''··„ „... ·'
„·'' „''··„ ¸·'' ),
„·'' '', ''··„ „·´ ',
'; ,''··...··'' `, ',
,' ,' Ñ맧 ; ',§Ãž
Functions
Function is the minimal capability to allow code reuse.
Aim: write a function that computes the temperature in kelvin from fahrenheit.
# starts with define keyword, tells the interpreter to not execute immediately since this is a
# function, then signature > name of function and then one argument
def fahr_to_kelvin(temp):
return ( ( temp - 32 ) * ( 5 / 9 ) ) + 273.15
print 'freezing point of water: ', fahr_to_kelvin(32)
BUGGGGGG: hint check the individual calculations
Types
Every variable has a type. Types are for example numbers (integer, float) or strings.
Python uses duck typing. The function print for example does not mind if a variable is once a string and later on used with a number. It will print whatever the variable is define last.
Other languages are stricter about types.
Integers are represented by binaries, can store more numbers.
But they represent only whole numbers (e.g. 1 2 3 4). Integers are represented as 32 bits.
At some point this runs out. There is an upper and lower limit for integers.
Floats (e.g. 0.43434) are bigger in memory their individual values are represented as integers.
Combining integer and float calculation is not possible as we did it before.
Debugging:
input_value = 212
print "212 - 32:", ( input_value - 32 ) # correct
print "(212 -32)*(5/9):" , (input_value-32)*(5/9) # gives 0
print 5/9 # gives 0 this calculation is a integer calculation, thus this the integer result is 0
# to force the use of float:
print float(5)/9
print 5/float(9) #at leaset one of the two has to be defined as float
print 5./9
# or use
from __future__ import division
# to force python to always use floats for divisions
Functions:
~ if argument needed >=3 times, better to write a function
responsibility of a function: it's nice if the function has one specific role (e.g. loading data)
Code refactoring
Boolean Operators and if, else and elif statement
# check if data in <filename> contains
# irregular minima (add up to 0) or
# if it contains triangle like maxima
def detectProblems( filename ):
"""
Here is the docstring that is called by the help with help(detectProblems)
check if data in <filename> contains irregular
minima (add up to 0) or if it contains triangle like maxima
"""
data = np.loadtxt( fname = filename , delimiter = ',' )
sum_min_per_day = data.min( axis = 0 ).sum() #sum is immediately calculated
# Boolean expression has only two outcomes: true or false
# compare if variable sum_min_per_day equals 0 then condition is true
if sum_min_per_day == 0:
print "Warning: Minima add up to 0!"
# if condition above is not met check against the the next condition
elif data.max( axis = 0 ) [0] == 0 and data.max( axis = 0 ) [20] == 20:
print "Warning: Suspicious Maxima!"
# else: prints if condition above is not met
else:
print "Everything is OK!"
help(detectProblems)
detectProblems( "inflammation-11.csv" )
detectProblems( "inflammation-04.csv" )
"""
text
""" to insert a multiline comment, docstrings
help(functionname) returns the docstrings
using default input
def detectProblems(filename, delim=","):
function can be called with or without giving the delimiter as second argument
="," - sets default argument
# to call the help for any function:
help(function name)
?np.loadtxt
numpy docs can be found at:
http://docs.scipy.org/doc/
LUNCH BREAK
.·´`·.
.·´ .·´··.
.·´ .·´ .·´`·.
.·´ .·´ .·´ `·.
.·´ `·..·´ `·.
.·´ .·´`·. `·. .·´`·.`·.
.·´ .·´ `·. `·. .·´ .·´ .·´·..
.·´·:· .·´ `·. `·. `·.·´ .·´ .·´ `·.
`·..·´ `·. `·..·´ .·´ .·´
`·. `·. `·´ .·´
`·. `·. .·´
`·. .·´ .·´ ~Galactus
`·. .·´ .·´
`·. .·´ .·´
`·.·´ ·:· .·´
`·..·´Æ¨‹
feedback, June 17, 12pm:
(red)
- a bit fast
- seating improper for teaching
- more exercises!
- background to data set?
- noise in room while started to talk
- didn't like the fires (not essential)
- too cold
- still not able to compose own program
- more time to understand!
- forgot more than 50% of the stuff learned yesterday
- how should I know how to call functions
(green)
- tab completion good
- now know where to look 4 help
- thanks for patience in explaining
- good to learn unknown bits/piece
- no confusion
- good stuff, +3
- 'x' vs "x"
- good speed, +2
- clear
- possibility to continue?
- amount of code
- best practise(s)
How to write your own programs
The final step...
Aim: Python program that gives a sequence of numbers that are the mean of the datasets
head -4 inflammation-01.csv | python readings.py --mean
sys library
sys.argv - a list of arguments - shell equivalent "$@"
python docs and much more can be found at:
https://www.python.org/
arith.py
import sys
# load operator function
import operator
# defines function arith
# loads arguments from command line
# depending on the operator carries
# out the calculation
def arith():
"""
function arith loads arguments from command line
and depending on the operator carries
out the calculation
"""
script = sys.argv[0]
# load arguments from command line
first = int( sys.argv[1] )
second = int( sys.argv[3] )
# provide a dictornary for the different operators
ops = {"+": operator.add,"-": operator.sub,"*": operator.mul,"/": operator.div}
# choose correct operator
op_func = ops[sys.argv[2]]
# carry out operation with first and second integer
calc = op_func(first, second)
# print the answer
print first, sys.argv[2], second," = ", calc
# calls function arith
arith()
execute:
python ./arith.py 1 + 2
readings-02.py
import sys
import numpy as np
def analyze( filename, action ):
data = np.loadtxt( filename, delimiter = "," )
container = None
if action == "--max":
container = data.max( axis = 1 )
elif action == "--min":
container = data.min( axis = 1 )
elif action == "--mean":
container = data.mean( axis = 1 )
for m in container:
print m
def main():
script = sys.argv[0]
action = sys.argv[1]
filename = sys.argv[2:]
for f in filename:
print filename
analyze( f, action)
main()
execute:
python ./readings-02.py --means <filename>
Version control system
git: tracks and saves changes
1. Configure git
# Username
git config --global user.name "name"
# Email
git config --global user.email "email@email.de"
# Color scheme
git config --global color.ui "auto"
# Define editor to be used by git
git config --global core.editor "nano"
# initialise git repository
git init
# history
git log
-> shows the identifier, author, date and title of commits
# compare the current file to the repository
git diff
# compare the staging area to the repository
git diff --staged
git diff HEAD~1 mars.txt
Which file can I track?
Potentially all fils. BUT it works best with txt based files.
figures: vector graphics
when tracking binary files it is very hard to follow changes.
# ignoring files: create a .gitignore file in git repository and within this file
# specify the files you want to ignore
# view the ignored files
git status --ignored
For those reading here I have now enabled the course evaluation at:
https://indico.mpi-cbg.de/event/6/evaluation/evaluate
Please fill it in before your leave today :)
..... .....
. ´ ` ·.::::::.· ´ ` .
.´ .:::@:::::) ::::: (:::::@:::. `.
: (::´ .:::::::. `::) :
.::`... .....:::::::::::::.... ...´::.
.::::::::::::::::::::::::::::::::::::::::::::::::::::::::.
.:::/::::::::::::::::::::::::::::::::::::::::::::::::::::::\:::.
::(--\::::::::::::::::::::::::::::::::::::::::::::::::::::::/--)::
`.:::::.``··::::::::::::::::::::::::::::::::::::::::··´´.:::::.´
`·:::::. ``··::::::::::::::::::::··´´ .:::::·´
`·:::::. `. .´ .:::::·´
`·:::::. ` ···· ´ .:::::·´
`·:::::. .:::::·´
.::::`·:::::.. ..:::::·´::::.
.:::::::::::::``·:::::::::::::·´´:::::::::::::.
.:::::::·´.::::::::::`::::::::´:::::::::::.`·:::::::.
·´´´´´ .:::::::::·´`·:::::::::::·´`·::::::::. `````·
.::::::·´ `·:::::·´ `·::::::.
·´´´´´ `·´ `````· §Ãž
told us to use version control, so we go for git.
more python material:
Coding Challenges and practice:
http://coderbyte.com/CodingArea/Challenges/
Explanation and challenges: http://introtopython.org/
Nice book in our library:
Practical Computing For Biologists
http://practicalcomputing.org/
Nice book for learning python (very approachable as a ... biologist)
http://pythonforbiologists.com/index.php/introduction-to-python-for-biologists/
Don't know if the free ebook is still on the website, here's my dropbox link:
https://www.dropbox.com/s/ri3l5sfhkettdik/p4b_r200.pdf?dl=0
Common python errors:
http://i.imgur.com/WRuJV6rl.png