Differences

This shows you the differences between two versions of the page.

--- python_notes [2019/12/11 13:21] – juckins
+++ python_notes [2023/01/25 10:27] – juckins
@@ Line 1: / Line 1: @@
-==== Python Notes ====
+==== Python Install Notes ====
+__11 Jan 2023__
+  # python3 -m pip list --outdated
+  Package             Version         Latest          Type
+  ------------------- --------------- --------------- -----
+  certifi             2022.9.24       2022.12.7       wheel
+  chromedriver-binary 108.0.5359.22.0 109.0.5414.74.0 sdist
+  dictor              0.1.10          0.1.11          sdist
+  pdf2image           1.16.0          1.16.2          wheel
+  pytesseract         0.3.8           0.3.9           wheel
+  pytz                2022.6          2022.7          wheel
+Upgraded all of the above except for pytesseract.  Then:
+  # find /usr -name "chromedriver" -exec ls -l {} \;
+  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
+__30 Nov 2022__
+Upgraded several python packages on physical and virtual servers.
+__26 Oct 2022__
+Upgraded several python packages on physical server.
+Current list of python modules:
+  [root@tidewater: ~]# python3 -m pip list
+  Package             Version
+  ------------------- ---------------
+  astral              3.0
+  backports.zoneinfo  0.2.1
+  black               22.8.0
+  certifi             2022.9.24
+  chardet             5.0.0
+  charset-normalizer  3.0.0
+  chromedriver        2.24.1
+  chromedriver-binary 107.0.5304.62.0
+  click               8.0.4
+  dataclasses         0.8
+  dictor              0.1.10
+  idna                3.4
+  importlib-metadata  4.8.3
+  importlib-resources 5.4.0
+  mypy-extensions     0.4.3
+  numpy               1.19.5
+  pathspec            0.9.0
+  pdf2image           1.16.0
+  Pillow              8.4.0
+  pip                 21.3.1
+  platformdirs        2.4.0
+  PyMySQL             1.0.2
+  pytesseract         0.3.8
+  pytz                2022.5
+  requests            2.27.1
+  selenium            3.141.0
+  setuptools          59.6.0
+  tomli               1.2.3
+  typed-ast           1.5.4
+  typing_extensions   4.1.1
+  urllib3             1.26.12
+  vboxapi             1.0
+  wget                3.2
+  zipp                3.6.0
+__30 Sep 2022__
+Current list of python modules:
+  [root@tidewater: ~]# python3 -m pip list
+  Package             Version
+  ------------------- ---------------
+  astral              2.2
+  black               22.8.0
+  certifi             2022.9.14
+  chardet             5.0.0
+  charset-normalizer  2.1.1
+  chromedriver        2.24.1
+  chromedriver-binary 105.0.5195.52.0
+  click               8.0.4
+  dataclasses         0.8
+  dictor              0.1.10
+  idna                3.4
+  importlib-metadata  4.8.3
+  mypy-extensions     0.4.3
+  numpy               1.19.5
+  pathspec            0.9.0
+  pdf2image           1.16.0
+  Pillow              8.4.0
+  pip                 21.3.1
+  platformdirs        2.4.0
+  PyMySQL             1.0.2
+  pytesseract         0.3.8
+  pytz                2022.2.1
+  requests            2.27.1
+  selenium            3.141.0
+  setuptools          59.6.0
+  tomli               1.2.3
+  typed-ast           1.5.4
+  typing_extensions   4.1.1
+  urllib3             1.26.12
+  vboxapi             1.0
+  wget                3.2
+  zipp                3.6.0
+__22 Sep 2022__
+Upgraded several python packages on both physical and virtual servers.
+__2 Aug 2022__
+Upgraded several python packages on both physical and virtual servers.
+__22 June 2022__
+Upgraded several python packages on both physical and virtual servers.
+__26 May 2022__
+Upgraded several python packages on both physical and virtual servers.
+__27 Apr 2022__
+Upgraded several python packages on both physical and virtual servers.
+__30 Mar 2022__
+Upgraded several python packages on both physical and virtual servers.
+__3 Mar 2022__
+Upgraded several python packages on both __physical__ and __virtual__ servers.
+  [root@tidewater: ~]# python3 -m pip list --outdated
+  Package             Version        Latest         Type
+  ------------------- -------------- -------------- -----
+  charset-normalizer  2.0.11         2.0.12         wheel
+  chromedriver-binary 98.0.4758.48.0 99.0.4844.51.0 sdist
+  numpy               1.12.1         1.19.5         wheel
+  pytesseract         0.3.8          0.3.9          wheel
+  [root@tidewater: ~]# python3 -m pip install --upgrade <package_name>
+Fix chromedriver-binary permissions:
+  [root@tidewater: ~]# find /usr -name "chromedriver" -exec ls -l {} \;
+  [root@tidewater: ~]# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
+The virtual server updates included the following:
+  [root@centos8S-vtatdb2-l: ~]# python3 -m pip list --outdated
+  Package            Version  Latest   Type
+  ------------------ -------- -------- -----
+  charset-normalizer 2.0.11   2.0.12   wheel
+  dbus-python        1.2.4    1.2.18   sdist
+  ethtool            0.14     0.15     sdist
+  httplib2           0.20.2   0.20.4   wheel
+  imageio            2.14.1   2.16.0   wheel
+  lxml               4.7.1    4.8.0    wheel
+  ntplib             0.3.3    0.4.0    wheel
+  pycairo            1.16.3   1.20.1   sdist
+  pycups             1.9.72   2.0.1    sdist
+  pycurl             7.43.0.2 7.44.1   sdist
+  PyGObject          3.28.3   3.42.0   sdist
+  pykickstart        3.35     3.37     sdist
+  python-augeas      0.5.0    1.1.0    sdist
+  pyudev             0.23.1   0.23.2   wheel
+  simpleline         1.1.1    1.9.0    wheel
+  slip               0.6.4    20191113 sdist
+  typing_extensions  4.0.1    4.1.1    wheel
+__2 Feb 2022__
+Upgraded several python packages on both __physical__ and __virtual__ servers.
+  [root@tidewater: ~]# python3 -m pip list --outdated
+  Package             Version        Latest         Type
+  ------------------- -------------- -------------- -----
+  charset-normalizer  2.0.10         2.0.11         wheel
+  chromedriver-binary 96.0.4664.35.0 98.0.4758.48.0 sdist
+  dictor              0.1.7          0.1.9          sdist
+  [root@tidewater: ~]# python3 -m pip install --upgrade <package_name>
+Fix chromedriver-binary permissions:
+  [root@tidewater: ~]# find /usr -name "chromedriver" -exec ls -l {} \;
+  [root@tidewater: ~]# chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
+The virtual server updates included the following:
+  [root@centos8S-vtatdb2-l: ~]# python3 -m pip list --outdated
+  Package         Version     Latest   Type
+  --------------- ----------- -------- -----
+  beautifulsoup4  4.6.3       4.10.0   wheel
+  blivet          3.4.0       3.4.3    wheel
+  chardet         3.0.4       4.0.0    wheel
+  coverage        4.5.1       6.2      wheel
+  cssselect       0.9.2       1.1.0    wheel
+  dasbus          1.2         1.6      wheel
+  dbus-python     1.2.4       1.2.18   sdist
+  decorator       4.2.1       5.1.1    wheel
+  dictor          0.1.7       0.1.9    sdist
+  dnspython       1.15.0      2.2.0    wheel
+  ethtool         0.14        0.15     sdist
+  html5lib        0.999999999 1.1      wheel
+  httplib2        0.18.1      0.20.2   wheel
+  humanize        0.5.1       3.14.0   wheel
+  idna            2.5         3.3      wheel
+  iniparse        0.4         0.5      wheel
+  langtable       0.0.51      0.0.57   wheel
+  lxml            4.2.3       4.7.1    wheel
+  ntplib          0.3.3       0.4.0    wheel
+  numpy           1.14.3      1.19.5   wheel
+  ordered-set     2.0.2       4.0.2    sdist
+  pexpect         4.3.1       4.8.0    wheel
+  pid             2.1.1       3.0.4    wheel
+  ply             3.9         3.11     wheel
+  productmd       1.11        1.33     wheel
+  psutil          5.4.3       5.9.0    wheel
+  ptyprocess      0.5.2       0.7.0    wheel
+  pycairo         1.16.3      1.20.1   sdist
+  pycups          1.9.72      2.0.1    sdist
+  pycurl          7.43.0.2    7.44.1   sdist
+  PyGObject       3.28.3      3.42.0   sdist
+  pykickstart     3.16.14     3.35     sdist
+  pyparsing       2.1.10      3.0.7    wheel
+  PySocks         1.6.8       1.7.1    wheel
+  python-augeas   0.5.0       1.1.0    sdist
+  python-dateutil 2.6.1       2.8.2    wheel
+  pytz            2017.2      2021.3   wheel
+  pyudev          0.21.0      0.23.1   wheel
+  PyYAML          3.12        6.0      wheel
+  requests        2.20.0      2.27.1   wheel
+  requests-file   1.4.3       1.5.1    wheel
+  scipy           1.0.0       1.5.4    wheel
+  setuptools      39.2.0      59.6.0   wheel
+  simpleline      1.1.1       1.9.0    wheel
+  six             1.11.0      1.16.0   wheel
+  slip            0.6.4       20191113 sdist
+  urllib3         1.24.2      1.26.8   wheel
+  wheel           0.36.2      0.37.1   wheel
+Most installed as user root; a few installed as non-admin user.
+These were unable to be updated:
+  $ python3 -m pip list --outdated
+  Package     Version  Latest   Type
+  ----------- -------- -------- -----
+  dbus-python 1.2.4    1.2.18   sdist
+  ethtool     0.14     0.15     sdist
+  pycairo     1.16.3   1.20.1   sdist
+  pycups      1.9.72   2.0.1    sdist
+  PyGObject   3.28.3   3.42.0   sdist
+  SLIP        20171205 20191113 sdist
+__20 Jan 2022__
+Upgraded several python packages.
+  [root@tidewater: ~]# python3 -m pip list --outdated
+  Package             Version        Latest         Type
+  ------------------- -------------- -------------- -----
+  charset-normalizer  2.0.7          2.0.10         wheel
+  chromedriver-binary 96.0.4664.35.0 98.0.4758.48.0 sdist
+  requests            2.26.0         2.27.1         wheel
+  setuptools          59.1.1         59.6.0         wheel
+  urllib3             1.26.7         1.26.8         wheel
+  [root@tidewater: ~]# python3 -m pip install --upgrade <package_name>
+Note: Did not update chromedriver-binary as Chrome is still version 97.0.4692.99
+__16 Nov 2021__
+Upgraded several python packages.
+  [root@tidewater: ~]# python3 -m pip list --outdated
+  Package             Version        Latest         Type
+  ------------------- -------------- -------------- -----
+  certifi             2021.5.30      2021.10.8      wheel
+  charset-normalizer  2.0.6          2.0.7          wheel
+  chromedriver-binary 94.0.4606.61.0 96.0.4664.35.0 sdist
+  idna                3.2            3.3            wheel
+  Pillow              8.3.2          8.4.0          wheel
+  pytz                2021.1         2021.3         wheel
+  setuptools          58.1.0         59.1.1         wheel
+  [root@tidewater: ~]# python3 -m pip install --upgrade <package_name>
+__29 Sep 2021__
+Upgraded the following python3 packages using the commands listed below:
+  pip
+  charset-normalizer
+  setuptools
+  urllib3
+  chromedriver-binary
+Then got the following error message for chromedriver-binary:
+  selenium.common.exceptions.SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 95
+  Current browser version is 94.0.4606.61 with binary path /usr/bin/google-chrome
+So did the following to fix:
+  python3 -m pip install chromedriver-binary==94.0.4606.61
+  find /usr -name "chromedriver" -exec ls -l {} \;
+  chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
+__21 Jul 2021__
+  # python3 -m pip list --outdated
+  Package             Version        Latest         Type
+  ------------------- -------------- -------------- -----
+  certifi             2020.12.5      2021.5.30      wheel
+  chromedriver-binary 91.0.4472.19.0 92.0.4515.43.0 sdist
+  idna                3.1            3.2            wheel
+  requests            2.25.1         2.26.0         wheel
+  setuptools          57.0.0         57.4.0         wheel
+  urllib3             1.26.5         1.26.6         wheel
+Upgraded all using:
+  python3 -m pip install --upgrade <package_name>
+Note on chromedriver-binary...don't forget to check/adjust permissions:
+  # find /usr -name "chromedriver" -exec ls -l {} \;
+  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
+__26 May 2021__
+These packages were outdated:
+  # python3 -m pip list --outdated
+  Package             Version        Latest         Type
+  ------------------- -------------- -------------- -----
+  chromedriver-binary 90.0.4430.24.0 91.0.4472.19.0 sdist
+  pip                 21.1.1         21.1.2         wheel
+  setuptools          56.0.0         57.0.0         wheel
+  WARNING: You are using pip version 21.1.1; however, version 21.1.2 is available.
+  You should consider upgrading via the '/bin/python3 -m pip install --upgrade pip' command.
+Upgraded all using:
+  python3 -m pip install --upgrade <package_name>
+__15 Apr 2021__
+Updated various python packages, including chromedriver-binary
+__3 Mar 2021__
+These packages were outdated:
+  # python3 -m pip list --outdated
+  Package             Version        Latest         Type
+  ------------------- -------------- -------------- -----
+  certifi             2020.11.8      2020.12.5      wheel
+  chardet             3.0.4          4.0.0          wheel
+  chromedriver-binary 87.0.4280.20.0 89.0.4389.23.0 sdist
+  httplib2            0.18.1         0.19.0         wheel
+  idna                2.10           3.1            wheel
+  PyMySQL             0.10.1         1.0.2          wheel
+  pytz                2020.4         2021.1         wheel
+  requests            2.25.0         2.25.1         wheel
+  selenium-wire       2.1.2          4.2.0          wheel
+  setuptools          50.3.2         54.0.0         wheel
+  urllib3             1.26.2         1.26.3         wheel
+  wheel               0.35.1         0.36.2         wheel
+Upgraded all using:
+  python3 -m pip install --upgrade <package_name>
+<color #ed1c24>except for PyMySQL.  'tidewater' needs updates to mariadb v10.</color>  Need to check python scripts first for ASMAD (and test on VM first).
+__18 Nov 2020__
+Manually updated chrome to v87.
+Ran the following:
+  python3 -m pip list --outdated
+With the resulting list, then ran the following for each <package_name>
+  python3 -m pip install --upgrade <package_name>
+__11 Oct 2020__
+To upgrade pip modules manually when dependencies give an error:
+  # python3 -m pip --list outdated
+  # python3 -m pip install --upgrade <package1> <package2>
+These are reported as outdated:
+  [root: ~]# python3 -m pip list --outdated
+  Package       Version    Latest    Type
+  ------------- ---------- --------- -----
+  certifi       2020.4.5.1 2020.6.20 wheel
+  idna          2.9        2.10      wheel
+  requests      2.23.0     2.24.0    wheel
+  selenium-wire 1.0.12     2.1.1     wheel
+  setuptools    46.4.0     50.3.0    wheel
+  urllib3       1.25.9     1.25.10   wheel
+All upgraded using the command:
+  python3 -m pip install --upgrade <package_name>
+__9 Oct 2020__
+Updated Chrome via yum to version 86.0.4240.75-1 and now check_envoy.pl fails.
+Did the following as root (checked on the VM "CENTOS78-test" first) to fix:
+  # /usr/bin/python3 -m pip uninstall chromedriver-binary
+  # /usr/bin/python3 -m pip install chromedriver-binary
+Note: you can also do specific versions of chromedrive-binary (see available versions at https://pypi.org/project/chromedriver-py/#history):
+  # python3 -m pip install chromedriver-binary==90.0.4430.24.0
+  # /usr/bin/python3 -m pip install --upgrade pip
+  # find /usr -name "chromedriver" -exec ls -l {} \;
+  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
+__6 Oct 2020__
+  # python3 -m pip install demjson
+  # python3 -m pip install wheel
+__26 Aug 2020__
+Updated Chrome via yum to version 85.0.4183.83-1 and now check_envoy.pl fails.
+Did the following as root to fix:
+  # /usr/bin/python3 -m pip uninstall chromedriver-binary
+  # /usr/bin/python3 -m pip install chromedriver-binary
+  # /usr/bin/python3 -m pip install --upgrade pip
+  # find /usr -name "chromedriver" -exec ls -l {} \;
+  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
+Re-ran the cron and script worked.
+__20 May 2020__
+Updated Chrome via yum to version 83.0.4103.61-1 and now check_envoy.pl fails.
+Did the following as root to fix:
+  # /usr/local/bin/pip3 uninstall chromedriver-binary
+  # /usr/local/bin/pip3 install chromedriver-binary
+  # /usr/bin/python3 -m pip install --upgrade pip
+  # find /usr -name "chromedriver" -exec ls -l {} \;
+  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
+Re-ran the cron and script worked.
+Then tried looking at old packages needing an update:
+  # cd /usr/local/bin
+  # ./pip3 list --outdated
+  # pip_upgrade_outdated
+__8 April 2020__
+Upgraded Chrome via yum to version 81.0.4044.92 and now check_envoy.pl fails.
+Did the following as root to fix:
+  # /usr/local/bin/pip3 uninstall chromedriver-binary
+  # /usr/local/bin/pip3 install chromedriver-binary
+  # /usr/local/bin/pip3 install pip-upgrade-outdated
+  # cd /usr/local/bin
+  # ./pip3 install --upgrade
+  # ./pip3 list --outdated
+  # pip_upgrade_outdated
+  # find /usr -name "chromedriver" -exec ls -l {} \;
+  /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
+  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
+The pip upgrade moved the astral package to v2.1 which broke sun.py and it needed some major changes.  Had to work through the [[https://astral.readthedocs.io/en/latest/index.html|Astral]] package docs.
 __11 December 2019__
-Upgraded Chrome via yum and selenium, chromedriver-binary fails.
+Upgraded Chrome via yum to version 79.0.3945.79 and now check_envoy.pl fails.
-Did the following as root:
+Did the following as root to fix:
-  /usr/local/bin/pip3 uninstall chromedriver-binary
+  # /usr/local/bin/pip3 uninstall chromedriver-binary
-  /usr/local/bin/pip3 install chromedriver-binary
+  # /usr/local/bin/pip3 install chromedriver-binary
+  # /usr/local/bin/pip3 install pip-upgrade-outdated
+  # pip-upgrade-outdated
+  # find /usr -name "chromedriver" -exec ls -l {} \;
+  /usr/lib/python3.6/site-packages/chromedriver
+  /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
+  # chmod 755 /usr/local/lib/python3.6/site-packages/chromedriver_binary/chromedriver
+Note that both **/usr/bin/pip3.6** and **/usr/local/bin/pip3.6** exist on the system now...needs to be fixed.
+----
 With CentOS7.7, python3 can load from base.  Do the following to fix previously-installed IUS release:
@@ Line 33: / Line 540: @@
   pip3.6 install --upgrade httplib2
   pip3.6 install pymysql
+  pip3.6 install selenium
   pip3.6 install selenium-wire
   pip3.6 install chromedriver
@@ Line 39: / Line 547: @@
   pip3.6 install wget
   pip3.6 install pip-upgrade-outdated
+  pip3.6 install pytz
+  pip3.6 install astral
+Note: replace the "pip3.6 install" with "/usr/bin/python3 -m pip install"
 To update all pip packages:
@@ Line 67: / Line 579: @@
   $ pip list
-[[https://docs.python.org/3/howto/logging.html|Logging in python]]
-  logging.basicConfig(format='%(asctime)s %(message)s', filename=log_file, level=logging.DEBUG)
-  logging.debug('This message should go to the log file')
-  logging.info('So should this')
-  logging.warning('And this, too')
-[[https://realpython.com/python-pep8/|Write beautiful python code with PEP-8]]