Christopher Juckins

SysAdmin Tips, Tricks and other Software Tools

User Tools

Site Tools

Trace:
python_notes

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
Next revisionBoth sides next revision
python_notes [2017/08/29 08:36] – created juckinspython_notes [2022/01/20 10:44] – juckins
Line 1: Line 1:
-==== Python Notes ====+==== Python Install Notes ====
  
-Use "pip" to manage python packages.  +__20 Jan 2022__
  
-  # yum install python2-pip+Upgraded several python packages. 
 + 
 +  [root@tidewater: ~]# python3 -m pip list --outdated 
 +  Package             Version        Latest         Type 
 +  ------------------- -------------- -------------- ----- 
 +  charset-normalizer  2.0.7          2.0.10         wheel 
 +  chromedriver-binary 96.0.4664.35.0 98.0.4758.48.0 sdist 
 +  requests            2.26.0         2.27.1         wheel 
 +  setuptools          59.1.1         59.6.0         wheel 
 +  urllib3             1.26.7         1.26.8         wheel 
 + 
 +  [root@tidewater: ~]# python3 -m pip install --upgrade <package_name> 
 + 
 +Note: Did not update chromedriver-binary as Chrome is still v97.0.4692.99 
 + 
 +__16 Nov 2021__
      
-  # pip list+Upgraded several python packages.
      
 +  [root@tidewater: ~]# python3 -m pip list --outdated
 +  Package             Version        Latest         Type
 +  ------------------- -------------- -------------- -----
 +  certifi             2021.5.30      2021.10.8      wheel
 +  charset-normalizer  2.0.6          2.0.7          wheel
 +  chromedriver-binary 94.0.4606.61.0 96.0.4664.35.0 sdist
 +  idna                3.2            3.3            wheel
 +  Pillow              8.3.2          8.4.0          wheel
 +  pytz                2021.1         2021.3         wheel
 +  setuptools          58.1.0         59.1.1         wheel
 +  
 +  [root@tidewater: ~]# python3 -m pip install --upgrade <package_name>
  
 +__29 Sep 2021__
 +
 +Upgraded the following python3 packages using the commands listed below:
 +
 +  pip
 +  charset-normalizer
 +  setuptools
 +  urllib3
 +  chromedriver-binary
 +  
 +Then got the following error message for chromedriver-binary:
 +
 +  selenium.common.exceptions.SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 95
 +  Current browser version is 94.0.4606.61 with binary path /usr/bin/google-chrome
 +
 +So did the following to fix:
 +
 +  python3 -m pip install chromedriver-binary==94.0.4606.61
 +  find /usr -name "chromedriver" -exec ls -l {} \;
 +  chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
 +
 +__21 Jul 2021__
 +
 +  # python3 -m pip list --outdated
 +  Package             Version        Latest         Type
 +  ------------------- -------------- -------------- -----
 +  certifi             2020.12.5      2021.5.30      wheel
 +  chromedriver-binary 91.0.4472.19.0 92.0.4515.43.0 sdist
 +  idna                3.1            3.2            wheel
 +  requests            2.25.1         2.26.0         wheel
 +  setuptools          57.0.0         57.4.0         wheel
 +  urllib3             1.26.5         1.26.6         wheel
 +
 +Upgraded all using:
 +
 +  python3 -m pip install --upgrade <package_name>
 +  
 +Note on chromedriver-binary...don't forget to check/adjust permissions:
 +
 +  # find /usr -name "chromedriver" -exec ls -l {} \;
 +  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
 +
 +__26 May 2021__
 +
 +
 +These packages were outdated:
 +
 +  # python3 -m pip list --outdated
 +  Package             Version        Latest         Type
 +  ------------------- -------------- -------------- -----
 +  chromedriver-binary 90.0.4430.24.0 91.0.4472.19.0 sdist
 +  pip                 21.1.1         21.1.2         wheel
 +  setuptools          56.0.0         57.0.0         wheel
 +  WARNING: You are using pip version 21.1.1; however, version 21.1.2 is available.
 +  You should consider upgrading via the '/bin/python3 -m pip install --upgrade pip' command.
 +
 +Upgraded all using:
 +
 +  python3 -m pip install --upgrade <package_name>
 +  
 +
 +__15 Apr 2021__
 +
 +Updated various python packages, including chromedriver-binary
 +
 +__3 Mar 2021__
 +
 +
 +These packages were outdated:
 +  
 +  # python3 -m pip list --outdated
 +  Package             Version        Latest         Type
 +  ------------------- -------------- -------------- -----
 +  certifi             2020.11.8      2020.12.5      wheel
 +  chardet             3.0.4          4.0.0          wheel
 +  chromedriver-binary 87.0.4280.20.0 89.0.4389.23.0 sdist
 +  httplib2            0.18.1         0.19.0         wheel
 +  idna                2.10           3.1            wheel
 +  PyMySQL             0.10.1         1.0.2          wheel
 +  pytz                2020.4         2021.1         wheel
 +  requests            2.25.0         2.25.1         wheel
 +  selenium-wire       2.1.2          4.2.0          wheel
 +  setuptools          50.3.2         54.0.0         wheel
 +  urllib3             1.26.2         1.26.3         wheel
 +  wheel               0.35.1         0.36.2         wheel
 +
 +Upgraded all using:
 +
 +  python3 -m pip install --upgrade <package_name>
 +  
 +<color #ed1c24>except for PyMySQL.  'tidewater' needs updates to mariadb v10.</color>  Need to check python scripts first for ASMAD (and test on VM first).
 +
 +__18 Nov 2020__
 +
 +Manually updated chrome to v87.
 +Ran the following:
 +
 +  python3 -m pip list --outdated
 +  
 +With the resulting list, then ran the following for each <package_name>
 +  python3 -m pip install --upgrade <package_name>
 +
 +__11 Oct 2020__
 +
 +To upgrade pip modules manually when dependencies give an error:
 +  # python3 -m pip --list outdated
 +  # python3 -m pip install --upgrade <package1> <package2> 
 +  
 +These are reported as outdated:
 +
 +  [root: ~]# python3 -m pip list --outdated
 +  Package       Version    Latest    Type
 +  ------------- ---------- --------- -----
 +  certifi       2020.4.5.1 2020.6.20 wheel
 +  idna          2.9        2.10      wheel
 +  requests      2.23.0     2.24.0    wheel
 +  selenium-wire 1.0.12     2.1.1     wheel
 +  setuptools    46.4.0     50.3.0    wheel
 +  urllib3       1.25.9     1.25.10   wheel
 +  
 +All upgraded using the command:
 +
 +  python3 -m pip install --upgrade <package_name>
 +
 +
 +__9 Oct 2020__
 +
 +Updated Chrome via yum to version 86.0.4240.75-1 and now check_envoy.pl fails.
 +
 +Did the following as root (checked on the VM "CENTOS78-test" first) to fix:
 +
 +  # /usr/bin/python3 -m pip uninstall chromedriver-binary
 +  # /usr/bin/python3 -m pip install chromedriver-binary
 +  
 +Note: you can also do specific versions of chromedrive-binary:
 +
 +  # python3 -m pip install chromedriver-binary==90.0.4430.24.0
 +
 +  # /usr/bin/python3 -m pip install --upgrade pip
 +  # find /usr -name "chromedriver" -exec ls -l {} \;
 +  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
 +
 +__6 Oct 2020__
 +  # python3 -m pip install demjson
 +  # python3 -m pip install wheel
 +
 +__26 Aug 2020__
 +
 +Updated Chrome via yum to version 85.0.4183.83-1 and now check_envoy.pl fails.
 +
 +Did the following as root to fix:
 +
 +  # /usr/bin/python3 -m pip uninstall chromedriver-binary
 +  # /usr/bin/python3 -m pip install chromedriver-binary
 +  # /usr/bin/python3 -m pip install --upgrade pip
 +  # find /usr -name "chromedriver" -exec ls -l {} \;
 +  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
 +
 +Re-ran the cron and script worked.
 +
 +__20 May 2020__
 +
 +Updated Chrome via yum to version 83.0.4103.61-1 and now check_envoy.pl fails.
 +
 +Did the following as root to fix:
 +
 +  # /usr/local/bin/pip3 uninstall chromedriver-binary
 +  # /usr/local/bin/pip3 install chromedriver-binary
 +  # /usr/bin/python3 -m pip install --upgrade pip
 +  # find /usr -name "chromedriver" -exec ls -l {} \;
 +  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
 +
 +Re-ran the cron and script worked.
 +
 +Then tried looking at old packages needing an update:
 +
 +  # cd /usr/local/bin
 +  # ./pip3 list --outdated
 +  # pip_upgrade_outdated
 +
 +__8 April 2020__
 +
 +Upgraded Chrome via yum to version 81.0.4044.92 and now check_envoy.pl fails.
 +
 +Did the following as root to fix:
 +
 +  # /usr/local/bin/pip3 uninstall chromedriver-binary
 +  # /usr/local/bin/pip3 install chromedriver-binary
 +  # /usr/local/bin/pip3 install pip-upgrade-outdated
 +  
 +  # cd /usr/local/bin
 +  # ./pip3 install --upgrade
 +  # ./pip3 list --outdated
 +  # pip_upgrade_outdated
 +  
 +  # find /usr -name "chromedriver" -exec ls -l {} \;
 +  
 +  /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
 +  
 +  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
 +
 +The pip upgrade moved the astral package to v2.1 which broke sun.py and it needed some major changes.  Had to work through the [[https://astral.readthedocs.io/en/latest/index.html|Astral]] package docs.
 +  
 +__11 December 2019__
 +
 +Upgraded Chrome via yum to version 79.0.3945.79 and now check_envoy.pl fails.
 +
 +Did the following as root to fix:
 +
 +  # /usr/local/bin/pip3 uninstall chromedriver-binary
 +  # /usr/local/bin/pip3 install chromedriver-binary
 +  # /usr/local/bin/pip3 install pip-upgrade-outdated
 +  # pip-upgrade-outdated
 +  
 +  # find /usr -name "chromedriver" -exec ls -l {} \;
 +  
 +  /usr/lib/python3.6/site-packages/chromedriver
 +  /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
 +
 +  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
 +
 +Note that both **/usr/bin/pip3.6** and **/usr/local/bin/pip3.6** exist on the system now...needs to be fixed.
 +
 +
 +
 +----
 +
 +With CentOS7.7, python3 can load from base.  Do the following to fix previously-installed IUS release:
 +
 +  # yum remove python36u
 +  # yum install python3
 +  # yum remove ius-release
 +
 +  * Then follow instructions for installing local python packages to ensure all still work.  
 +  * If you have pip problems, uninstall/reinstall using instructions at https://pip.pypa.io/en/stable/installing/
 +  * Ensure pip3.6 is in your $PATH or you have a sym link to it (/usr/local/bin) in /usr/bin
 +
 +__Install Python 3.6 on CentOS 7__:
 +  * https://linuxhint.com/install_python3_centos7/
 +  * https://janikarhunen.fi/how-to-install-python-3-6-1-on-centos-7.html
 +
 +Do some local configuration for my scripts (su - root):
 +  pip3.6 install --upgrade pip
 +  pip3.6 install --upgrade httplib2
 +  pip3.6 install pymysql
 +  pip3.6 install selenium
 +  pip3.6 install selenium-wire
 +  pip3.6 install chromedriver
 +  pip3.6 install chromedriver_binary
 +  pip3.6 install dictor
 +  pip3.6 install wget
 +  pip3.6 install pip-upgrade-outdated
 +  pip3.6 install pytz
 +  pip3.6 install astral
 +
 +Note: replace the "pip3.6 install" with "/usr/bin/python3 -m pip install"
 +  
 +To update all pip packages:
 +
 +  # pip_upgrade_outdated
 +
 +Note that https://pypi.org/project/pip-upgrade-outdated/ is a good resource to update all PIP packages.
 +  
 +Download and install geckodriver (see https://github.com/mozilla/geckodriver/releases)
 +
 +As root, copy geckodriver to /usr/local/bin and ensure permissions are 755
 +
 +Notes at https://stackoverflow.com/questions/40208051/selenium-using-python-geckodriver-executable-needs-to-be-in-path
 +  
 +Notes about [[https://www.tutorialspoint.com/python3/python_database_access.htm|Python database access]]
 +
 +Python3.7 on Fedora 28:
 +  * dnf install python37
 +  * wget https://bootstrap.pypa.io/get-pip.py
 +  * python3.7 get-pip.py
 +  * python3.7 -m pip install requests
 +
 +Use "pip" to manage python packages:
 +
 +  # yum install python2-pip
 +
 +To list installed python modules:
 +  
 +  $ pip list
python_notes.txt · Last modified: 2023/04/05 08:56 by juckins