Python Install Notes
5 Apr 2023
# python3 -m pip list --outdated Package Version Latest Type ------------------- --------------- --------------- ----- chromedriver-binary 110.0.5481.30.0 112.0.5615.28.0 sdist mypy-extensions 0.4.3 1.0.0 wheel pdf2image 1.16.2 1.16.3 wheel pytesseract 0.3.8 0.3.9 wheel pytz 2022.7.1 2023.3 wheel urllib3 1.26.14 1.26.15 wheel
Ran this:
# python3 -m pip install --upgrade <package_name>
# find /usr -name "chromedriver" -exec ls -l {} \;
# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
11 Jan 2023
# python3 -m pip list --outdated Package Version Latest Type ------------------- --------------- --------------- ----- certifi 2022.9.24 2022.12.7 wheel chromedriver-binary 108.0.5359.22.0 109.0.5414.74.0 sdist dictor 0.1.10 0.1.11 sdist pdf2image 1.16.0 1.16.2 wheel pytesseract 0.3.8 0.3.9 wheel pytz 2022.6 2022.7 wheel
Upgraded all of the above except for pytesseract. Then:
# find /usr -name "chromedriver" -exec ls -l {} \;
# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
30 Nov 2022
Upgraded several python packages on physical and virtual servers.
26 Oct 2022
Upgraded several python packages on physical server.
Current list of python modules:
[root@tidewater: ~]# python3 -m pip list Package Version ------------------- --------------- astral 3.0 backports.zoneinfo 0.2.1 black 22.8.0 certifi 2022.9.24 chardet 5.0.0 charset-normalizer 3.0.0 chromedriver 2.24.1 chromedriver-binary 107.0.5304.62.0 click 8.0.4 dataclasses 0.8 dictor 0.1.10 idna 3.4 importlib-metadata 4.8.3 importlib-resources 5.4.0 mypy-extensions 0.4.3 numpy 1.19.5 pathspec 0.9.0 pdf2image 1.16.0 Pillow 8.4.0 pip 21.3.1 platformdirs 2.4.0 PyMySQL 1.0.2 pytesseract 0.3.8 pytz 2022.5 requests 2.27.1 selenium 3.141.0 setuptools 59.6.0 tomli 1.2.3 typed-ast 1.5.4 typing_extensions 4.1.1 urllib3 1.26.12 vboxapi 1.0 wget 3.2 zipp 3.6.0
30 Sep 2022
Current list of python modules:
[root@tidewater: ~]# python3 -m pip list Package Version ------------------- --------------- astral 2.2 black 22.8.0 certifi 2022.9.14 chardet 5.0.0 charset-normalizer 2.1.1 chromedriver 2.24.1 chromedriver-binary 105.0.5195.52.0 click 8.0.4 dataclasses 0.8 dictor 0.1.10 idna 3.4 importlib-metadata 4.8.3 mypy-extensions 0.4.3 numpy 1.19.5 pathspec 0.9.0 pdf2image 1.16.0 Pillow 8.4.0 pip 21.3.1 platformdirs 2.4.0 PyMySQL 1.0.2 pytesseract 0.3.8 pytz 2022.2.1 requests 2.27.1 selenium 3.141.0 setuptools 59.6.0 tomli 1.2.3 typed-ast 1.5.4 typing_extensions 4.1.1 urllib3 1.26.12 vboxapi 1.0 wget 3.2 zipp 3.6.0
22 Sep 2022
Upgraded several python packages on both physical and virtual servers.
2 Aug 2022
Upgraded several python packages on both physical and virtual servers.
22 June 2022
Upgraded several python packages on both physical and virtual servers.
26 May 2022
Upgraded several python packages on both physical and virtual servers.
27 Apr 2022
Upgraded several python packages on both physical and virtual servers.
30 Mar 2022
Upgraded several python packages on both physical and virtual servers.
3 Mar 2022
Upgraded several python packages on both physical and virtual servers.
[root@tidewater: ~]# python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- charset-normalizer 2.0.11 2.0.12 wheel chromedriver-binary 98.0.4758.48.0 99.0.4844.51.0 sdist numpy 1.12.1 1.19.5 wheel pytesseract 0.3.8 0.3.9 wheel
[root@tidewater: ~]# python3 -m pip install --upgrade <package_name>
Fix chromedriver-binary permissions:
[root@tidewater: ~]# find /usr -name "chromedriver" -exec ls -l {} \;
[root@tidewater: ~]# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
The virtual server updates included the following:
[root@centos8S-vtatdb2-l: ~]# python3 -m pip list --outdated Package Version Latest Type ------------------ -------- -------- ----- charset-normalizer 2.0.11 2.0.12 wheel dbus-python 1.2.4 1.2.18 sdist ethtool 0.14 0.15 sdist httplib2 0.20.2 0.20.4 wheel imageio 2.14.1 2.16.0 wheel lxml 4.7.1 4.8.0 wheel ntplib 0.3.3 0.4.0 wheel pycairo 1.16.3 1.20.1 sdist pycups 1.9.72 2.0.1 sdist pycurl 7.43.0.2 7.44.1 sdist PyGObject 3.28.3 3.42.0 sdist pykickstart 3.35 3.37 sdist python-augeas 0.5.0 1.1.0 sdist pyudev 0.23.1 0.23.2 wheel simpleline 1.1.1 1.9.0 wheel slip 0.6.4 20191113 sdist typing_extensions 4.0.1 4.1.1 wheel
2 Feb 2022
Upgraded several python packages on both physical and virtual servers.
[root@tidewater: ~]# python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- charset-normalizer 2.0.10 2.0.11 wheel chromedriver-binary 96.0.4664.35.0 98.0.4758.48.0 sdist dictor 0.1.7 0.1.9 sdist
[root@tidewater: ~]# python3 -m pip install --upgrade <package_name>
Fix chromedriver-binary permissions:
[root@tidewater: ~]# find /usr -name "chromedriver" -exec ls -l {} \;
[root@tidewater: ~]# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
The virtual server updates included the following:
[root@centos8S-vtatdb2-l: ~]# python3 -m pip list --outdated Package Version Latest Type --------------- ----------- -------- ----- beautifulsoup4 4.6.3 4.10.0 wheel blivet 3.4.0 3.4.3 wheel chardet 3.0.4 4.0.0 wheel coverage 4.5.1 6.2 wheel cssselect 0.9.2 1.1.0 wheel dasbus 1.2 1.6 wheel dbus-python 1.2.4 1.2.18 sdist decorator 4.2.1 5.1.1 wheel dictor 0.1.7 0.1.9 sdist dnspython 1.15.0 2.2.0 wheel ethtool 0.14 0.15 sdist html5lib 0.999999999 1.1 wheel httplib2 0.18.1 0.20.2 wheel humanize 0.5.1 3.14.0 wheel idna 2.5 3.3 wheel iniparse 0.4 0.5 wheel langtable 0.0.51 0.0.57 wheel lxml 4.2.3 4.7.1 wheel ntplib 0.3.3 0.4.0 wheel numpy 1.14.3 1.19.5 wheel ordered-set 2.0.2 4.0.2 sdist pexpect 4.3.1 4.8.0 wheel pid 2.1.1 3.0.4 wheel ply 3.9 3.11 wheel productmd 1.11 1.33 wheel psutil 5.4.3 5.9.0 wheel ptyprocess 0.5.2 0.7.0 wheel pycairo 1.16.3 1.20.1 sdist pycups 1.9.72 2.0.1 sdist pycurl 7.43.0.2 7.44.1 sdist PyGObject 3.28.3 3.42.0 sdist pykickstart 3.16.14 3.35 sdist pyparsing 2.1.10 3.0.7 wheel PySocks 1.6.8 1.7.1 wheel python-augeas 0.5.0 1.1.0 sdist python-dateutil 2.6.1 2.8.2 wheel pytz 2017.2 2021.3 wheel pyudev 0.21.0 0.23.1 wheel PyYAML 3.12 6.0 wheel requests 2.20.0 2.27.1 wheel requests-file 1.4.3 1.5.1 wheel scipy 1.0.0 1.5.4 wheel setuptools 39.2.0 59.6.0 wheel simpleline 1.1.1 1.9.0 wheel six 1.11.0 1.16.0 wheel slip 0.6.4 20191113 sdist urllib3 1.24.2 1.26.8 wheel wheel 0.36.2 0.37.1 wheel
Most installed as user root; a few installed as non-admin user.
These were unable to be updated:
$ python3 -m pip list --outdated Package Version Latest Type ----------- -------- -------- ----- dbus-python 1.2.4 1.2.18 sdist ethtool 0.14 0.15 sdist pycairo 1.16.3 1.20.1 sdist pycups 1.9.72 2.0.1 sdist PyGObject 3.28.3 3.42.0 sdist SLIP 20171205 20191113 sdist
20 Jan 2022
Upgraded several python packages.
[root@tidewater: ~]# python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- charset-normalizer 2.0.7 2.0.10 wheel chromedriver-binary 96.0.4664.35.0 98.0.4758.48.0 sdist requests 2.26.0 2.27.1 wheel setuptools 59.1.1 59.6.0 wheel urllib3 1.26.7 1.26.8 wheel
[root@tidewater: ~]# python3 -m pip install --upgrade <package_name>
Note: Did not update chromedriver-binary as Chrome is still version 97.0.4692.99
16 Nov 2021
Upgraded several python packages.
[root@tidewater: ~]# python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- certifi 2021.5.30 2021.10.8 wheel charset-normalizer 2.0.6 2.0.7 wheel chromedriver-binary 94.0.4606.61.0 96.0.4664.35.0 sdist idna 3.2 3.3 wheel Pillow 8.3.2 8.4.0 wheel pytz 2021.1 2021.3 wheel setuptools 58.1.0 59.1.1 wheel [root@tidewater: ~]# python3 -m pip install --upgrade <package_name>
29 Sep 2021
Upgraded the following python3 packages using the commands listed below:
pip charset-normalizer setuptools urllib3 chromedriver-binary
Then got the following error message for chromedriver-binary:
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 95 Current browser version is 94.0.4606.61 with binary path /usr/bin/google-chrome
So did the following to fix:
python3 -m pip install chromedriver-binary==94.0.4606.61
find /usr -name "chromedriver" -exec ls -l {} \;
chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
21 Jul 2021
# python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- certifi 2020.12.5 2021.5.30 wheel chromedriver-binary 91.0.4472.19.0 92.0.4515.43.0 sdist idna 3.1 3.2 wheel requests 2.25.1 2.26.0 wheel setuptools 57.0.0 57.4.0 wheel urllib3 1.26.5 1.26.6 wheel
Upgraded all using:
python3 -m pip install --upgrade <package_name>
Note on chromedriver-binary…don't forget to check/adjust permissions:
# find /usr -name "chromedriver" -exec ls -l {} \;
# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
26 May 2021
These packages were outdated:
# python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- chromedriver-binary 90.0.4430.24.0 91.0.4472.19.0 sdist pip 21.1.1 21.1.2 wheel setuptools 56.0.0 57.0.0 wheel WARNING: You are using pip version 21.1.1; however, version 21.1.2 is available. You should consider upgrading via the '/bin/python3 -m pip install --upgrade pip' command.
Upgraded all using:
python3 -m pip install --upgrade <package_name>
15 Apr 2021
Updated various python packages, including chromedriver-binary
3 Mar 2021
These packages were outdated:
# python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- certifi 2020.11.8 2020.12.5 wheel chardet 3.0.4 4.0.0 wheel chromedriver-binary 87.0.4280.20.0 89.0.4389.23.0 sdist httplib2 0.18.1 0.19.0 wheel idna 2.10 3.1 wheel PyMySQL 0.10.1 1.0.2 wheel pytz 2020.4 2021.1 wheel requests 2.25.0 2.25.1 wheel selenium-wire 2.1.2 4.2.0 wheel setuptools 50.3.2 54.0.0 wheel urllib3 1.26.2 1.26.3 wheel wheel 0.35.1 0.36.2 wheel
Upgraded all using:
python3 -m pip install --upgrade <package_name>
except for PyMySQL. 'tidewater' needs updates to mariadb v10. Need to check python scripts first for ASMAD (and test on VM first).
18 Nov 2020
Manually updated chrome to v87. Ran the following:
python3 -m pip list --outdated
With the resulting list, then ran the following for each <package_name>
python3 -m pip install --upgrade <package_name>
11 Oct 2020
To upgrade pip modules manually when dependencies give an error:
# python3 -m pip --list outdated # python3 -m pip install --upgrade <package1> <package2>
These are reported as outdated:
[root: ~]# python3 -m pip list --outdated Package Version Latest Type ------------- ---------- --------- ----- certifi 2020.4.5.1 2020.6.20 wheel idna 2.9 2.10 wheel requests 2.23.0 2.24.0 wheel selenium-wire 1.0.12 2.1.1 wheel setuptools 46.4.0 50.3.0 wheel urllib3 1.25.9 1.25.10 wheel
All upgraded using the command:
python3 -m pip install --upgrade <package_name>
9 Oct 2020
Updated Chrome via yum to version 86.0.4240.75-1 and now check_envoy.pl fails.
Did the following as root (checked on the VM "CENTOS78-test" first) to fix:
# /usr/bin/python3 -m pip uninstall chromedriver-binary # /usr/bin/python3 -m pip install chromedriver-binary
Note: you can also do specific versions of chromedrive-binary (see available versions at https://pypi.org/project/chromedriver-py/#history):
# python3 -m pip install chromedriver-binary==90.0.4430.24.0
# /usr/bin/python3 -m pip install --upgrade pip
# find /usr -name "chromedriver" -exec ls -l {} \;
# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
6 Oct 2020
# python3 -m pip install demjson # python3 -m pip install wheel
26 Aug 2020
Updated Chrome via yum to version 85.0.4183.83-1 and now check_envoy.pl fails.
Did the following as root to fix:
# /usr/bin/python3 -m pip uninstall chromedriver-binary
# /usr/bin/python3 -m pip install chromedriver-binary
# /usr/bin/python3 -m pip install --upgrade pip
# find /usr -name "chromedriver" -exec ls -l {} \;
# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
Re-ran the cron and script worked.
20 May 2020
Updated Chrome via yum to version 83.0.4103.61-1 and now check_envoy.pl fails.
Did the following as root to fix:
# /usr/local/bin/pip3 uninstall chromedriver-binary
# /usr/local/bin/pip3 install chromedriver-binary
# /usr/bin/python3 -m pip install --upgrade pip
# find /usr -name "chromedriver" -exec ls -l {} \;
# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
Re-ran the cron and script worked.
Then tried looking at old packages needing an update:
# cd /usr/local/bin # ./pip3 list --outdated # pip_upgrade_outdated
8 April 2020
Upgraded Chrome via yum to version 81.0.4044.92 and now check_envoy.pl fails.
Did the following as root to fix:
# /usr/local/bin/pip3 uninstall chromedriver-binary
# /usr/local/bin/pip3 install chromedriver-binary
# /usr/local/bin/pip3 install pip-upgrade-outdated
# cd /usr/local/bin
# ./pip3 install --upgrade
# ./pip3 list --outdated
# pip_upgrade_outdated
# find /usr -name "chromedriver" -exec ls -l {} \;
/usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
The pip upgrade moved the astral package to v2.1 which broke sun.py and it needed some major changes. Had to work through the Astral package docs.
11 December 2019
Upgraded Chrome via yum to version 79.0.3945.79 and now check_envoy.pl fails.
Did the following as root to fix:
# /usr/local/bin/pip3 uninstall chromedriver-binary
# /usr/local/bin/pip3 install chromedriver-binary
# /usr/local/bin/pip3 install pip-upgrade-outdated
# pip-upgrade-outdated
# find /usr -name "chromedriver" -exec ls -l {} \;
/usr/lib/python3.6/site-packages/chromedriver
/usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
Note that both /usr/bin/pip3.6 and /usr/local/bin/pip3.6 exist on the system now…needs to be fixed.
With CentOS7.7, python3 can load from base. Do the following to fix previously-installed IUS release:
# yum remove python36u # yum install python3 # yum remove ius-release
- Then follow instructions for installing local python packages to ensure all still work.
- If you have pip problems, uninstall/reinstall using instructions at https://pip.pypa.io/en/stable/installing/
- Ensure pip3.6 is in your $PATH or you have a sym link to it (/usr/local/bin) in /usr/bin
Install Python 3.6 on CentOS 7:
Do some local configuration for my scripts (su - root):
pip3.6 install --upgrade pip pip3.6 install --upgrade httplib2 pip3.6 install pymysql pip3.6 install selenium pip3.6 install selenium-wire pip3.6 install chromedriver pip3.6 install chromedriver_binary pip3.6 install dictor pip3.6 install wget pip3.6 install pip-upgrade-outdated pip3.6 install pytz pip3.6 install astral
Note: replace the "pip3.6 install" with "/usr/bin/python3 -m pip install"
To update all pip packages:
# pip_upgrade_outdated
Note that https://pypi.org/project/pip-upgrade-outdated/ is a good resource to update all PIP packages.
Download and install geckodriver (see https://github.com/mozilla/geckodriver/releases)
As root, copy geckodriver to /usr/local/bin and ensure permissions are 755
Notes about Python database access
Python3.7 on Fedora 28:
- dnf install python37
- python3.7 get-pip.py
- python3.7 -m pip install requests
Use "pip" to manage python packages:
# yum install python2-pip
To list installed python modules:
$ pip list