前言
本文介绍了如何快速搭建一个pypiserver,通过自建pypiserver,我们可以解决网络环境不好,或者离线无法安装python包的问题。如果结合最新的GitLab CI/CD和pipenv我相信各位还可以玩出更多的花样。
pypiserver - minimal PyPI server for use with pip/easy_install
更新记录
2018年04月12日 - 初稿
阅读原文 - https://wsgzao.github.io/post/pypiserver/
扩展阅读
pypiserver - https://github.com/pypiserver/pypiserver
pypiserver 简介
pypiserver is a minimal PyPI compatible server for pip or easy_install. It is based on bottle and serves packages from regular directories. Wheels, bdists, eggs and accompanying PGP-signatures can be uploaded either with pip, setuptools, twine, pypi-uploader, or simply copied with scp.
pypiserver 服务端配置
如果你的Linux环境缺少Python 2.7可以参考我的文章直接离线升级至最新版本
Python 2.6 升级至 Python 2.7 的实践心得
https://wsgzao.github.io/post/python-2-6-update-to-2-7/
pypiserver > 1.2.x works with python 2.7 and 3.3+ or pypy. Older python-versions may still work, but they are not tested. For legacy python versions, use pypiserver-1.1.x series.
# 替换pip为阿里云,感概豆瓣的时代已经过去tee ~/.pip/pip.conf <<-'EOF'[global] index-url = https://mirrors.aliyun.com/pypi/simple/ [install] trusted-host= mirrors.aliyun.com EOF# 直接在线安装pypiserverpip install pypiserver# 离线下载pypiservermkdir /tmp/pypiserver pip install -d /tmp/pypiserver/ pypiserver# Copy packages into this directory.mkdir ~/packages# Copy some packages into your ~/packages folder and then get your pypiserver up and running:pypi-server -p 8080 ~/packages &
pypiserver 客户端配置
## Download and Install hosted packages.pip install --extra-index-url http://localhost:8080/simple/ ...# orpip install --extra-index-url http://localhost:8080## Search hosted packagespip search --index http://localhost:8080 ...# 个人推荐的配置tee ~/.pip/pip.conf <<-'EOF'[global] index-url = http://172.28.70.126/simple extra-index-url = https://mirrors.aliyun.com/pypi/simple/ [install] trusted-host = 172.28.70.126 EOF
pypiserver 进阶配置
pypiserver Running as a systemd service
https://github.com/pypiserver/pypiserver#running-as-a-systemd-service
Adjusting the paths and adding this file as pypiserver.service into your systemd/system directory will allow management of the pypiserver process with systemctl, e.g. systemctl start pypiserver.
More useful information about systemd can be found at https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units
# 安装需要的包yum install nginx -y pip install passlib pypiserver gunicorn# 创建pypiserver服务方便服务启停管理tee /usr/lib/systemd/system/pypiserver.service <<-'EOF'[Unit] Description=gunicorn daemon After=network.target [Service] PIDFile=/run/pypiserver.pid ExecStart=/usr/local/bin/gunicorn -w16 \ --pid /run/pypiserver.pid \ -b :10012 \ 'pypiserver:app(root="/var/www/pypi")'ExecReload=/bin/kill -s HUP $MAINPIDExecStop=/bin/kill -s TERM $MAINPID[Install] WantedBy=multi-user.target EOF# Warning: pypiserver.service changed on disk. Run 'systemctl daemon-reload' to reload units.systemctl daemon-reload# 启动pypiserver服务systemctl enable pypiserver.service systemctl start pypiserver.service systemctl status pypiserver.service# 停止pypiserver服务systemctl disable pypiserver.service systemctl stop pypiserver.service systemctl status pypiserver.service [root@centos7 run]# systemctl status pypiserver.service pypiserver.service - gunicorn daemon Loaded: loaded (/usr/lib/systemd/system/pypiserver.service; disabled; vendor preset: disabled) Active: active (running) since Fri 2018-04-13 15:14:08 CST; 859ms ago Main PID: 22524 (gunicorn) CGroup: /system.slice/pypiserver.service ├─22524 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi") ├─22530 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi") ├─22531 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi") ├─22532 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi") ├─22533 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi") ├─22534 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi") ├─22535 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi") ├─22536 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi") ├─22537 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi") ├─22538 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi") ├─22539 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi") └─22540 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi") Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22531] [INFO] Booting worker with pid: 22531 Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22532] [INFO] Booting worker with pid: 22532 Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22533] [INFO] Booting worker with pid: 22533 Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22534] [INFO] Booting worker with pid: 22534 Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22535] [INFO] Booting worker with pid: 22535 Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22536] [INFO] Booting worker with pid: 22536 Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22537] [INFO] Booting worker with pid: 22537 Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22538] [INFO] Booting worker with pid: 22538 Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22539] [INFO] Booting worker with pid: 22539 Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22540] [INFO] Booting worker with pid: 22540 [root@centos7 run]# netstat -tnlpActive Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 1517/dnsmasq tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 977/sshd tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 978/cupsd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1383/master tcp 0 0 127.0.0.1:6011 0.0.0.0:* LISTEN 19378/sshd: root@pt tcp 0 0 0.0.0.0:10012 0.0.0.0:* LISTEN 22524/python tcp6 0 0 :::111 :::* LISTEN 1/systemd tcp6 0 0 :::22 :::* LISTEN 977/sshd tcp6 0 0 ::1:631 :::* LISTEN 978/cupsd tcp6 0 0 ::1:25 :::* LISTEN 1383/master tcp6 0 0 ::1:6011 :::* LISTEN 19378/sshd: root@pt # 检查pypiserver服务cd /var/www/pypi# 向仓库中添加python package[root@centos7 pypi]# pip download pypiserverCollecting pypiserver Downloading https://mirrors.aliyun.com/pypi/packages/d7/78/5772432dad2b9e754ab92f4d301fa507069b9decc8c943c1b18c2043ff4f/pypiserver-1.2.1-py2.py3-none-any.whl (83kB) 100% |████████████████████████████████| 92kB 643kB/s Saved ./pypiserver-1.2.1-py2.py3-none-any.whl Successfully downloaded pypiserver [root@centos7 pypi]# lltotal 84 -rw-r--r-- 1 root root 83529 Apr 13 14:55 pypiserver-1.2.1-py2.py3-none-any.whl# 搜索刚才下载的package[root@centos7 pypi]# pip search -i http://127.0.0.1:10012 pypiserverpypiserver (1.2.1) - 1.2.1 INSTALLED: 1.2.1 (latest)# 配置nginx做反向代理tee /etc/nginx/conf.d/pypi.conf <<-'EOF'upstream pypiserver { server 127.0.0.1:10012; } server { listen 10087; # disable any limits to avoid HTTP 413 for large package uploads client_max_body_size 0; location / { proxy_pass http://pypiserver/; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # When setting up pypiserver behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings. proxy_set_header X-Forwarded-Proto $scheme; proxy_buffering off; proxy_request_buffering off; } location /packages/ { alias /var/www/pypi; # static file } } EOF# 启动nginxsystemctl enable nginx systemctl start nginx systemctl status nginx# 检查nginx服务pip search -i http://172.28.79.126:10087 pypiserver pypiserver (1.2.1) - 1.2.1 INSTALLED: 1.2.1 (latest)
作者:王奥OX
链接:https://www.jianshu.com/p/9cb6f698caa3