==== Python Install Notes ==== __5 Apr 2023__ # python3 -m pip list --outdated Package Version Latest Type ------------------- --------------- --------------- ----- chromedriver-binary 110.0.5481.30.0 112.0.5615.28.0 sdist mypy-extensions 0.4.3 1.0.0 wheel pdf2image 1.16.2 1.16.3 wheel pytesseract 0.3.8 0.3.9 wheel pytz 2022.7.1 2023.3 wheel urllib3 1.26.14 1.26.15 wheel Ran this: # python3 -m pip install --upgrade # find /usr -name "chromedriver" -exec ls -l {} \; # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver __11 Jan 2023__ # python3 -m pip list --outdated Package Version Latest Type ------------------- --------------- --------------- ----- certifi 2022.9.24 2022.12.7 wheel chromedriver-binary 108.0.5359.22.0 109.0.5414.74.0 sdist dictor 0.1.10 0.1.11 sdist pdf2image 1.16.0 1.16.2 wheel pytesseract 0.3.8 0.3.9 wheel pytz 2022.6 2022.7 wheel Upgraded all of the above except for pytesseract. Then: # find /usr -name "chromedriver" -exec ls -l {} \; # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver __30 Nov 2022__ Upgraded several python packages on physical and virtual servers. __26 Oct 2022__ Upgraded several python packages on physical server. Current list of python modules: [root@tidewater: ~]# python3 -m pip list Package Version ------------------- --------------- astral 3.0 backports.zoneinfo 0.2.1 black 22.8.0 certifi 2022.9.24 chardet 5.0.0 charset-normalizer 3.0.0 chromedriver 2.24.1 chromedriver-binary 107.0.5304.62.0 click 8.0.4 dataclasses 0.8 dictor 0.1.10 idna 3.4 importlib-metadata 4.8.3 importlib-resources 5.4.0 mypy-extensions 0.4.3 numpy 1.19.5 pathspec 0.9.0 pdf2image 1.16.0 Pillow 8.4.0 pip 21.3.1 platformdirs 2.4.0 PyMySQL 1.0.2 pytesseract 0.3.8 pytz 2022.5 requests 2.27.1 selenium 3.141.0 setuptools 59.6.0 tomli 1.2.3 typed-ast 1.5.4 typing_extensions 4.1.1 urllib3 1.26.12 vboxapi 1.0 wget 3.2 zipp 3.6.0 __30 Sep 2022__ Current list of python modules: [root@tidewater: ~]# python3 -m pip list Package Version ------------------- --------------- astral 2.2 black 22.8.0 certifi 2022.9.14 chardet 5.0.0 charset-normalizer 2.1.1 chromedriver 2.24.1 chromedriver-binary 105.0.5195.52.0 click 8.0.4 dataclasses 0.8 dictor 0.1.10 idna 3.4 importlib-metadata 4.8.3 mypy-extensions 0.4.3 numpy 1.19.5 pathspec 0.9.0 pdf2image 1.16.0 Pillow 8.4.0 pip 21.3.1 platformdirs 2.4.0 PyMySQL 1.0.2 pytesseract 0.3.8 pytz 2022.2.1 requests 2.27.1 selenium 3.141.0 setuptools 59.6.0 tomli 1.2.3 typed-ast 1.5.4 typing_extensions 4.1.1 urllib3 1.26.12 vboxapi 1.0 wget 3.2 zipp 3.6.0 __22 Sep 2022__ Upgraded several python packages on both physical and virtual servers. __2 Aug 2022__ Upgraded several python packages on both physical and virtual servers. __22 June 2022__ Upgraded several python packages on both physical and virtual servers. __26 May 2022__ Upgraded several python packages on both physical and virtual servers. __27 Apr 2022__ Upgraded several python packages on both physical and virtual servers. __30 Mar 2022__ Upgraded several python packages on both physical and virtual servers. __3 Mar 2022__ Upgraded several python packages on both __physical__ and __virtual__ servers. [root@tidewater: ~]# python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- charset-normalizer 2.0.11 2.0.12 wheel chromedriver-binary 98.0.4758.48.0 99.0.4844.51.0 sdist numpy 1.12.1 1.19.5 wheel pytesseract 0.3.8 0.3.9 wheel [root@tidewater: ~]# python3 -m pip install --upgrade Fix chromedriver-binary permissions: [root@tidewater: ~]# find /usr -name "chromedriver" -exec ls -l {} \; [root@tidewater: ~]# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver The virtual server updates included the following: [root@centos8S-vtatdb2-l: ~]# python3 -m pip list --outdated Package Version Latest Type ------------------ -------- -------- ----- charset-normalizer 2.0.11 2.0.12 wheel dbus-python 1.2.4 1.2.18 sdist ethtool 0.14 0.15 sdist httplib2 0.20.2 0.20.4 wheel imageio 2.14.1 2.16.0 wheel lxml 4.7.1 4.8.0 wheel ntplib 0.3.3 0.4.0 wheel pycairo 1.16.3 1.20.1 sdist pycups 1.9.72 2.0.1 sdist pycurl 7.43.0.2 7.44.1 sdist PyGObject 3.28.3 3.42.0 sdist pykickstart 3.35 3.37 sdist python-augeas 0.5.0 1.1.0 sdist pyudev 0.23.1 0.23.2 wheel simpleline 1.1.1 1.9.0 wheel slip 0.6.4 20191113 sdist typing_extensions 4.0.1 4.1.1 wheel __2 Feb 2022__ Upgraded several python packages on both __physical__ and __virtual__ servers. [root@tidewater: ~]# python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- charset-normalizer 2.0.10 2.0.11 wheel chromedriver-binary 96.0.4664.35.0 98.0.4758.48.0 sdist dictor 0.1.7 0.1.9 sdist [root@tidewater: ~]# python3 -m pip install --upgrade Fix chromedriver-binary permissions: [root@tidewater: ~]# find /usr -name "chromedriver" -exec ls -l {} \; [root@tidewater: ~]# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver The virtual server updates included the following: [root@centos8S-vtatdb2-l: ~]# python3 -m pip list --outdated Package Version Latest Type --------------- ----------- -------- ----- beautifulsoup4 4.6.3 4.10.0 wheel blivet 3.4.0 3.4.3 wheel chardet 3.0.4 4.0.0 wheel coverage 4.5.1 6.2 wheel cssselect 0.9.2 1.1.0 wheel dasbus 1.2 1.6 wheel dbus-python 1.2.4 1.2.18 sdist decorator 4.2.1 5.1.1 wheel dictor 0.1.7 0.1.9 sdist dnspython 1.15.0 2.2.0 wheel ethtool 0.14 0.15 sdist html5lib 0.999999999 1.1 wheel httplib2 0.18.1 0.20.2 wheel humanize 0.5.1 3.14.0 wheel idna 2.5 3.3 wheel iniparse 0.4 0.5 wheel langtable 0.0.51 0.0.57 wheel lxml 4.2.3 4.7.1 wheel ntplib 0.3.3 0.4.0 wheel numpy 1.14.3 1.19.5 wheel ordered-set 2.0.2 4.0.2 sdist pexpect 4.3.1 4.8.0 wheel pid 2.1.1 3.0.4 wheel ply 3.9 3.11 wheel productmd 1.11 1.33 wheel psutil 5.4.3 5.9.0 wheel ptyprocess 0.5.2 0.7.0 wheel pycairo 1.16.3 1.20.1 sdist pycups 1.9.72 2.0.1 sdist pycurl 7.43.0.2 7.44.1 sdist PyGObject 3.28.3 3.42.0 sdist pykickstart 3.16.14 3.35 sdist pyparsing 2.1.10 3.0.7 wheel PySocks 1.6.8 1.7.1 wheel python-augeas 0.5.0 1.1.0 sdist python-dateutil 2.6.1 2.8.2 wheel pytz 2017.2 2021.3 wheel pyudev 0.21.0 0.23.1 wheel PyYAML 3.12 6.0 wheel requests 2.20.0 2.27.1 wheel requests-file 1.4.3 1.5.1 wheel scipy 1.0.0 1.5.4 wheel setuptools 39.2.0 59.6.0 wheel simpleline 1.1.1 1.9.0 wheel six 1.11.0 1.16.0 wheel slip 0.6.4 20191113 sdist urllib3 1.24.2 1.26.8 wheel wheel 0.36.2 0.37.1 wheel Most installed as user root; a few installed as non-admin user. These were unable to be updated: $ python3 -m pip list --outdated Package Version Latest Type ----------- -------- -------- ----- dbus-python 1.2.4 1.2.18 sdist ethtool 0.14 0.15 sdist pycairo 1.16.3 1.20.1 sdist pycups 1.9.72 2.0.1 sdist PyGObject 3.28.3 3.42.0 sdist SLIP 20171205 20191113 sdist __20 Jan 2022__ Upgraded several python packages. [root@tidewater: ~]# python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- charset-normalizer 2.0.7 2.0.10 wheel chromedriver-binary 96.0.4664.35.0 98.0.4758.48.0 sdist requests 2.26.0 2.27.1 wheel setuptools 59.1.1 59.6.0 wheel urllib3 1.26.7 1.26.8 wheel [root@tidewater: ~]# python3 -m pip install --upgrade Note: Did not update chromedriver-binary as Chrome is still version 97.0.4692.99 __16 Nov 2021__ Upgraded several python packages. [root@tidewater: ~]# python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- certifi 2021.5.30 2021.10.8 wheel charset-normalizer 2.0.6 2.0.7 wheel chromedriver-binary 94.0.4606.61.0 96.0.4664.35.0 sdist idna 3.2 3.3 wheel Pillow 8.3.2 8.4.0 wheel pytz 2021.1 2021.3 wheel setuptools 58.1.0 59.1.1 wheel [root@tidewater: ~]# python3 -m pip install --upgrade __29 Sep 2021__ Upgraded the following python3 packages using the commands listed below: pip charset-normalizer setuptools urllib3 chromedriver-binary Then got the following error message for chromedriver-binary: selenium.common.exceptions.SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 95 Current browser version is 94.0.4606.61 with binary path /usr/bin/google-chrome So did the following to fix: python3 -m pip install chromedriver-binary==94.0.4606.61 find /usr -name "chromedriver" -exec ls -l {} \; chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver __21 Jul 2021__ # python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- certifi 2020.12.5 2021.5.30 wheel chromedriver-binary 91.0.4472.19.0 92.0.4515.43.0 sdist idna 3.1 3.2 wheel requests 2.25.1 2.26.0 wheel setuptools 57.0.0 57.4.0 wheel urllib3 1.26.5 1.26.6 wheel Upgraded all using: python3 -m pip install --upgrade Note on chromedriver-binary...don't forget to check/adjust permissions: # find /usr -name "chromedriver" -exec ls -l {} \; # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver __26 May 2021__ These packages were outdated: # python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- chromedriver-binary 90.0.4430.24.0 91.0.4472.19.0 sdist pip 21.1.1 21.1.2 wheel setuptools 56.0.0 57.0.0 wheel WARNING: You are using pip version 21.1.1; however, version 21.1.2 is available. You should consider upgrading via the '/bin/python3 -m pip install --upgrade pip' command. Upgraded all using: python3 -m pip install --upgrade __15 Apr 2021__ Updated various python packages, including chromedriver-binary __3 Mar 2021__ These packages were outdated: # python3 -m pip list --outdated Package Version Latest Type ------------------- -------------- -------------- ----- certifi 2020.11.8 2020.12.5 wheel chardet 3.0.4 4.0.0 wheel chromedriver-binary 87.0.4280.20.0 89.0.4389.23.0 sdist httplib2 0.18.1 0.19.0 wheel idna 2.10 3.1 wheel PyMySQL 0.10.1 1.0.2 wheel pytz 2020.4 2021.1 wheel requests 2.25.0 2.25.1 wheel selenium-wire 2.1.2 4.2.0 wheel setuptools 50.3.2 54.0.0 wheel urllib3 1.26.2 1.26.3 wheel wheel 0.35.1 0.36.2 wheel Upgraded all using: python3 -m pip install --upgrade except for PyMySQL. 'tidewater' needs updates to mariadb v10. Need to check python scripts first for ASMAD (and test on VM first). __18 Nov 2020__ Manually updated chrome to v87. Ran the following: python3 -m pip list --outdated With the resulting list, then ran the following for each python3 -m pip install --upgrade __11 Oct 2020__ To upgrade pip modules manually when dependencies give an error: # python3 -m pip --list outdated # python3 -m pip install --upgrade These are reported as outdated: [root: ~]# python3 -m pip list --outdated Package Version Latest Type ------------- ---------- --------- ----- certifi 2020.4.5.1 2020.6.20 wheel idna 2.9 2.10 wheel requests 2.23.0 2.24.0 wheel selenium-wire 1.0.12 2.1.1 wheel setuptools 46.4.0 50.3.0 wheel urllib3 1.25.9 1.25.10 wheel All upgraded using the command: python3 -m pip install --upgrade __9 Oct 2020__ Updated Chrome via yum to version 86.0.4240.75-1 and now check_envoy.pl fails. Did the following as root (checked on the VM "CENTOS78-test" first) to fix: # /usr/bin/python3 -m pip uninstall chromedriver-binary # /usr/bin/python3 -m pip install chromedriver-binary Note: you can also do specific versions of chromedrive-binary (see available versions at https://pypi.org/project/chromedriver-py/#history): # python3 -m pip install chromedriver-binary==90.0.4430.24.0 # /usr/bin/python3 -m pip install --upgrade pip # find /usr -name "chromedriver" -exec ls -l {} \; # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver __6 Oct 2020__ # python3 -m pip install demjson # python3 -m pip install wheel __26 Aug 2020__ Updated Chrome via yum to version 85.0.4183.83-1 and now check_envoy.pl fails. Did the following as root to fix: # /usr/bin/python3 -m pip uninstall chromedriver-binary # /usr/bin/python3 -m pip install chromedriver-binary # /usr/bin/python3 -m pip install --upgrade pip # find /usr -name "chromedriver" -exec ls -l {} \; # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver Re-ran the cron and script worked. __20 May 2020__ Updated Chrome via yum to version 83.0.4103.61-1 and now check_envoy.pl fails. Did the following as root to fix: # /usr/local/bin/pip3 uninstall chromedriver-binary # /usr/local/bin/pip3 install chromedriver-binary # /usr/bin/python3 -m pip install --upgrade pip # find /usr -name "chromedriver" -exec ls -l {} \; # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver Re-ran the cron and script worked. Then tried looking at old packages needing an update: # cd /usr/local/bin # ./pip3 list --outdated # pip_upgrade_outdated __8 April 2020__ Upgraded Chrome via yum to version 81.0.4044.92 and now check_envoy.pl fails. Did the following as root to fix: # /usr/local/bin/pip3 uninstall chromedriver-binary # /usr/local/bin/pip3 install chromedriver-binary # /usr/local/bin/pip3 install pip-upgrade-outdated # cd /usr/local/bin # ./pip3 install --upgrade # ./pip3 list --outdated # pip_upgrade_outdated # find /usr -name "chromedriver" -exec ls -l {} \; /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver The pip upgrade moved the astral package to v2.1 which broke sun.py and it needed some major changes. Had to work through the [[https://astral.readthedocs.io/en/latest/index.html|Astral]] package docs. __11 December 2019__ Upgraded Chrome via yum to version 79.0.3945.79 and now check_envoy.pl fails. Did the following as root to fix: # /usr/local/bin/pip3 uninstall chromedriver-binary # /usr/local/bin/pip3 install chromedriver-binary # /usr/local/bin/pip3 install pip-upgrade-outdated # pip-upgrade-outdated # find /usr -name "chromedriver" -exec ls -l {} \; /usr/lib/python3.6/site-packages/chromedriver /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver Note that both **/usr/bin/pip3.6** and **/usr/local/bin/pip3.6** exist on the system now...needs to be fixed. ---- With CentOS7.7, python3 can load from base. Do the following to fix previously-installed IUS release: # yum remove python36u # yum install python3 # yum remove ius-release * Then follow instructions for installing local python packages to ensure all still work. * If you have pip problems, uninstall/reinstall using instructions at https://pip.pypa.io/en/stable/installing/ * Ensure pip3.6 is in your $PATH or you have a sym link to it (/usr/local/bin) in /usr/bin __Install Python 3.6 on CentOS 7__: * https://linuxhint.com/install_python3_centos7/ * https://janikarhunen.fi/how-to-install-python-3-6-1-on-centos-7.html Do some local configuration for my scripts (su - root): pip3.6 install --upgrade pip pip3.6 install --upgrade httplib2 pip3.6 install pymysql pip3.6 install selenium pip3.6 install selenium-wire pip3.6 install chromedriver pip3.6 install chromedriver_binary pip3.6 install dictor pip3.6 install wget pip3.6 install pip-upgrade-outdated pip3.6 install pytz pip3.6 install astral Note: replace the "pip3.6 install" with "/usr/bin/python3 -m pip install" To update all pip packages: # pip_upgrade_outdated Note that https://pypi.org/project/pip-upgrade-outdated/ is a good resource to update all PIP packages. Download and install geckodriver (see https://github.com/mozilla/geckodriver/releases) As root, copy geckodriver to /usr/local/bin and ensure permissions are 755 Notes at https://stackoverflow.com/questions/40208051/selenium-using-python-geckodriver-executable-needs-to-be-in-path Notes about [[https://www.tutorialspoint.com/python3/python_database_access.htm|Python database access]] Python3.7 on Fedora 28: * dnf install python37 * wget https://bootstrap.pypa.io/get-pip.py * python3.7 get-pip.py * python3.7 -m pip install requests Use "pip" to manage python packages: # yum install python2-pip To list installed python modules: $ pip list