Tuesday, 6 December 2022

Selenium (Python) (Firefox) is unable to write Firefox profile. Permission denied (os error 13)

When I run my Python code using Selenium to open (and scrape) a website, using a profile parameter, I get the following error message:

    selenium.common.exceptions.SessionNotCreatedException: Message: Failed to set preferences: Unable to write Firefox profile: Permission denied (os error 13)

I use the following code:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
from selenium.webdriver.firefox.options import Options
from datetime import datetime

f = open("/home/username/Documents/AH/log2.txt", "a")
f.write('gestart')
f.close()

# I use the below code when I use crontab, read below..
#from pyvirtualdisplay import Display
#display = Display(visible=0, size=(2880, 1800)).start()

website = "https://www.ah.nl/"
options=Options()
options.add_argument('-profile')
options.add_argument(r'/home/username/.mozilla/firefox/n2n9yy4r.default-release') 

driver = webdriver.Firefox(options=options, service_log_path="/home/username/geckodriver.log")

driver.implicitly_wait(20)
driver.get(website)

tijd = WebDriverWait(driver,40).until(EC.presence_of_element_located((By.XPATH,"//*[@id='navigation-header']/div[1]/div[1]/div/div/p/a")))
tijdbezorging = tijd.text

driver.close()
driver.quit()

f = open("/home/username/Documents/AH/testlog.txt", "a")
f.write(tijdbezorging)
f.close()

The full error message when I run this code using python3 filenamecode.py:

SendAHStatus.py:46: DeprecationWarning: service_log_path has been deprecated, please pass in a Service object
  driver = webdriver.Firefox(options=options, service_log_path="/home/username/geckodriver.log")
/usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
Traceback (most recent call last):
  File "SendAHStatus.py", line 46, in <module>
    driver = webdriver.Firefox(options=options, service_log_path="/home/username/geckodriver.log")
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/firefox/webdriver.py", line 197, in __init__
    super().__init__(command_executor=executor, options=options, keep_alive=True)
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/webdriver.py", line 288, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/webdriver.py", line 381, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/webdriver.py", line 444, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/errorhandler.py", line 249, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: Failed to set preferences: Unable to write Firefox profile: Permission denied (os error 13)

The code DOES work when I schedule it using sudo crontab -e. Yes, using sudo. See code below:

SHELL=/bin/bash
PATH=/usr/local/bin/:usr/bin: and some paths
01 01 * * * python3 myscriptname.py

And, the code also works (via my command line) without the profile parameter. I also tried using a different profile path like, /home/username/profiledir. This also works via sudo crontab, but not via my regular command line.

--ADDED---

I created the profile in FireFox using about:profiles -> create new profile. From there I gave all users, incl. the user that I am using 'odroid', all permissions to this folder, pls see below:

enter image description here

---//ADDED------

Btw, I uncomment the virtual display when using cron.

I could run my code using sudo python3 myscript.py. But then I get this message:

Running Firefox as root in a regular user's session is not supported.  ($XAUTHORITY is /home/username/.Xauthority which is owned by username.)

When changing ownership of XAUTHORITY to root, I can run my script, but I don't want to go down that path.

What I can derive from all of this:

  • The webdriver is working,
  • Its a permission issue since the code runs using elevated permission, using cron.

Please help me with this, its getting frustrating.

My environment:

  • Selenium 4.6.0
  • Python 3.8
  • Firefox 107.0
  • Ubuntu 20.04.5 LTS

Your help is much appreciated!



from Selenium (Python) (Firefox) is unable to write Firefox profile. Permission denied (os error 13)

No comments:

Post a Comment