knitr - error when importing python module

I am having trouble when running the python engine in knitr. I can import some modules but not others. For example I can import numpy but not pandas.

{r, engine='python'} import pandas

I get the error.

Quitting from lines 50-51 (prepayment.Rmd) 
Error in (knit_engines$get(options$engine))(options) : 
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named pandas
Calls: <Anonymous> ... process_group.block -> call_block -> block_exec -> in_dir -> <Anonymous>
In addition: Warning message:
running command ''python'  -c 'import pandas' 2>&1' had status 1 
Execution halted

It must be something to do with system path and where I am pointing.

Normally in IPython i use

import sys
sys.path.append('/path/to/directory/')

To add paths. I don't know what the problem is.

from knitr - error when importing python module