Christopher Juckins

SysAdmin Tips, Tricks and other Software Tools

User Tools

Site Tools

Trace:
python_notes

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
Next revisionBoth sides next revision
python_notes [2019/06/08 10:05] – juckinspython_notes [2021/07/21 08:54] – juckins
Line 1: Line 1:
-==== Python Notes ====+==== Python Install Notes ==== 
 + 
 +__21 Jul 2021__ 
 + 
 +  # python3 -m pip list --outdated 
 +  Package             Version        Latest         Type 
 +  ------------------- -------------- -------------- ----- 
 +  certifi             2020.12.5      2021.5.30      wheel 
 +  chromedriver-binary 91.0.4472.19.0 92.0.4515.43.0 sdist 
 +  idna                3.1            3.2            wheel 
 +  requests            2.25.1         2.26.0         wheel 
 +  setuptools          57.0.0         57.4.0         wheel 
 +  urllib3             1.26.5         1.26.6         wheel 
 + 
 +Upgraded all using: 
 + 
 +  python3 -m pip install --upgrade <package_name>
      
-Install Python 3 on CentOS 7:+Note on chromedriver-binary...don't forget to check/adjust permissions: 
 + 
 +  # find /usr -name "chromedriver" -exec ls -l {} \; 
 +  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver 
 + 
 +__26 May 2021__ 
 + 
 + 
 +These packages were outdated: 
 + 
 +  # python3 -m pip list --outdated 
 +  Package             Version        Latest         Type 
 +  ------------------- -------------- -------------- ----- 
 +  chromedriver-binary 90.0.4430.24.0 91.0.4472.19.0 sdist 
 +  pip                 21.1.1         21.1.2         wheel 
 +  setuptools          56.0.0         57.0.0         wheel 
 +  WARNING: You are using pip version 21.1.1; however, version 21.1.2 is available. 
 +  You should consider upgrading via the '/bin/python3 -m pip install --upgrade pip' command. 
 + 
 +Upgraded all using: 
 + 
 +  python3 -m pip install --upgrade <package_name> 
 +   
 + 
 +__15 Apr 2021__ 
 + 
 +Updated various python packages, including chromedriver-binary 
 + 
 +__3 Mar 2021__ 
 + 
 + 
 +These packages were outdated: 
 +   
 +  # python3 -m pip list --outdated 
 +  Package             Version        Latest         Type 
 +  ------------------- -------------- -------------- ----- 
 +  certifi             2020.11.8      2020.12.5      wheel 
 +  chardet             3.0.4          4.0.0          wheel 
 +  chromedriver-binary 87.0.4280.20.0 89.0.4389.23.0 sdist 
 +  httplib2            0.18.1         0.19.0         wheel 
 +  idna                2.10           3.1            wheel 
 +  PyMySQL             0.10.1         1.0.2          wheel 
 +  pytz                2020.4         2021.1         wheel 
 +  requests            2.25.0         2.25.1         wheel 
 +  selenium-wire       2.1.2          4.2.0          wheel 
 +  setuptools          50.3.2         54.0.0         wheel 
 +  urllib3             1.26.2         1.26.3         wheel 
 +  wheel               0.35.1         0.36.2         wheel 
 + 
 +Upgraded all using: 
 + 
 +  python3 -m pip install --upgrade <package_name> 
 +   
 +<color #ed1c24>except for PyMySQL.  'tidewater' needs updates to mariadb v10.</color>  Need to check python scripts first for ASMAD (and test on VM first). 
 + 
 +__18 Nov 2020__ 
 + 
 +Manually updated chrome to v87. 
 +Ran the following: 
 + 
 +  python3 -m pip list --outdated 
 +   
 +With the resulting list, then ran the following for each <package_name> 
 +  python3 -m pip install --upgrade <package_name> 
 + 
 +__11 Oct 2020__ 
 + 
 +To upgrade pip modules manually when dependencies give an error: 
 +  # python3 -m pip --list outdated 
 +  # python3 -m pip install --upgrade <package1> <package2>  
 +   
 +These are reported as outdated: 
 + 
 +  [root: ~]# python3 -m pip list --outdated 
 +  Package       Version    Latest    Type 
 +  ------------- ---------- --------- ----- 
 +  certifi       2020.4.5.1 2020.6.20 wheel 
 +  idna          2.9        2.10      wheel 
 +  requests      2.23.0     2.24.0    wheel 
 +  selenium-wire 1.0.12     2.1.1     wheel 
 +  setuptools    46.4.0     50.3.0    wheel 
 +  urllib3       1.25.9     1.25.10   wheel 
 +   
 +All upgraded using the command: 
 + 
 +  python3 -m pip install --upgrade <package_name> 
 + 
 + 
 +__9 Oct 2020__ 
 + 
 +Updated Chrome via yum to version 86.0.4240.75-1 and now check_envoy.pl fails. 
 + 
 +Did the following as root (checked on the VM "CENTOS78-test" first) to fix: 
 + 
 +  # /usr/bin/python3 -m pip uninstall chromedriver-binary 
 +  # /usr/bin/python3 -m pip install chromedriver-binary 
 +   
 +Note: you can also do specific versions of chromedrive-binary: 
 + 
 +  # python3 -m pip install chromedriver-binary==90.0.4430.24.0 
 + 
 +  # /usr/bin/python3 -m pip install --upgrade pip 
 +  # find /usr -name "chromedriver" -exec ls -l {} \; 
 +  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver 
 + 
 +__6 Oct 2020__ 
 +  # python3 -m pip install demjson 
 +  # python3 -m pip install wheel 
 + 
 +__26 Aug 2020__ 
 + 
 +Updated Chrome via yum to version 85.0.4183.83-1 and now check_envoy.pl fails. 
 + 
 +Did the following as root to fix: 
 + 
 +  # /usr/bin/python3 -m pip uninstall chromedriver-binary 
 +  # /usr/bin/python3 -m pip install chromedriver-binary 
 +  # /usr/bin/python3 -m pip install --upgrade pip 
 +  # find /usr -name "chromedriver" -exec ls -l {} \; 
 +  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver 
 + 
 +Re-ran the cron and script worked. 
 + 
 +__20 May 2020__ 
 + 
 +Updated Chrome via yum to version 83.0.4103.61-1 and now check_envoy.pl fails. 
 + 
 +Did the following as root to fix: 
 + 
 +  # /usr/local/bin/pip3 uninstall chromedriver-binary 
 +  # /usr/local/bin/pip3 install chromedriver-binary 
 +  # /usr/bin/python3 -m pip install --upgrade pip 
 +  # find /usr -name "chromedriver" -exec ls -l {} \; 
 +  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver 
 + 
 +Re-ran the cron and script worked. 
 + 
 +Then tried looking at old packages needing an update: 
 + 
 +  # cd /usr/local/bin 
 +  # ./pip3 list --outdated 
 +  # pip_upgrade_outdated 
 + 
 +__8 April 2020__ 
 + 
 +Upgraded Chrome via yum to version 81.0.4044.92 and now check_envoy.pl fails. 
 + 
 +Did the following as root to fix: 
 + 
 +  # /usr/local/bin/pip3 uninstall chromedriver-binary 
 +  # /usr/local/bin/pip3 install chromedriver-binary 
 +  # /usr/local/bin/pip3 install pip-upgrade-outdated 
 +   
 +  # cd /usr/local/bin 
 +  # ./pip3 install --upgrade 
 +  # ./pip3 list --outdated 
 +  # pip_upgrade_outdated 
 +   
 +  # find /usr -name "chromedriver" -exec ls -l {} \; 
 +   
 +  /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver 
 +   
 +  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver 
 + 
 +The pip upgrade moved the astral package to v2.1 which broke sun.py and it needed some major changes.  Had to work through the [[https://astral.readthedocs.io/en/latest/index.html|Astral]] package docs. 
 +   
 +__11 December 2019__ 
 + 
 +Upgraded Chrome via yum to version 79.0.3945.79 and now check_envoy.pl fails. 
 + 
 +Did the following as root to fix: 
 + 
 +  # /usr/local/bin/pip3 uninstall chromedriver-binary 
 +  # /usr/local/bin/pip3 install chromedriver-binary 
 +  # /usr/local/bin/pip3 install pip-upgrade-outdated 
 +  # pip-upgrade-outdated 
 +   
 +  # find /usr -name "chromedriver" -exec ls -l {} \; 
 +   
 +  /usr/lib/python3.6/site-packages/chromedriver 
 +  /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver 
 + 
 +  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver 
 + 
 +Note that both **/usr/bin/pip3.6** and **/usr/local/bin/pip3.6** exist on the system now...needs to be fixed. 
 + 
 + 
 + 
 +---- 
 + 
 +With CentOS7.7, python3 can load from base.  Do the following to fix previously-installed IUS release: 
 + 
 +  # yum remove python36u 
 +  # yum install python3 
 +  # yum remove ius-release 
 + 
 +  * Then follow instructions for installing local python packages to ensure all still work.   
 +  * If you have pip problems, uninstall/reinstall using instructions at https://pip.pypa.io/en/stable/installing/ 
 +  * Ensure pip3.6 is in your $PATH or you have a sym link to it (/usr/local/bin) in /usr/bin 
 + 
 +__Install Python 3.6 on CentOS 7__:
   * https://linuxhint.com/install_python3_centos7/   * https://linuxhint.com/install_python3_centos7/
   * https://janikarhunen.fi/how-to-install-python-3-6-1-on-centos-7.html   * https://janikarhunen.fi/how-to-install-python-3-6-1-on-centos-7.html
 +
 +Do some local configuration for my scripts (su - root):
 +  pip3.6 install --upgrade pip
 +  pip3.6 install --upgrade httplib2
 +  pip3.6 install pymysql
 +  pip3.6 install selenium
 +  pip3.6 install selenium-wire
 +  pip3.6 install chromedriver
 +  pip3.6 install chromedriver_binary
 +  pip3.6 install dictor
 +  pip3.6 install wget
 +  pip3.6 install pip-upgrade-outdated
 +  pip3.6 install pytz
 +  pip3.6 install astral
 +
 +Note: replace the "pip3.6 install" with "/usr/bin/python3 -m pip install"
 +  
 +To update all pip packages:
 +
 +  # pip_upgrade_outdated
 +
 +Note that https://pypi.org/project/pip-upgrade-outdated/ is a good resource to update all PIP packages.
 +  
 +Download and install geckodriver (see https://github.com/mozilla/geckodriver/releases)
 +
 +As root, copy geckodriver to /usr/local/bin and ensure permissions are 755
 +
 +Notes at https://stackoverflow.com/questions/40208051/selenium-using-python-geckodriver-executable-needs-to-be-in-path
 +  
 +Notes about [[https://www.tutorialspoint.com/python3/python_database_access.htm|Python database access]]
  
 Python3.7 on Fedora 28: Python3.7 on Fedora 28:
python_notes.txt · Last modified: 2023/04/05 08:56 by juckins