flask – Uobis

03 Oct

SSL/TLS Certificates on Flask Application with Let’s Encrypt and NGINX

Let’s Encrypt is a new certificate authority (CA) offering free and automated SSL/TLS certificates. Certificates issued by Let’s Encrypt are trusted by most browsers in production today, including Internet Explorer on Windows Vista. Simply download and run the Let’s Encrypt client to generate a certificate.

(there are a few more steps than that, of course, though not many)
Step 1: Download LetsEncrypt

Install git if you haven’t done so yet:

# apt-get install git

Use git to get the application and store it somewhere (ie: /opt)

$ sudo git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt

Step 2: Webroot Plugin

The Webroot plugin works by placing a special file in the /.well-known directory within your document root, which can be opened (through your web server) by the Let’s Encrypt service for validation.
Depending on your configuration, you may need to explicitly allow access to the /.well-known directory.

location /.well-known {
alias /home/user/webapps/appname/.well-known;
}

Restart NGNIX

# sudo service nginx status

Step 3: Generate your certificate and Strong Diffie-Hellman Group

The first time you run the command below, you will be asked to provide an e-mail address to be associated to the domain or subdomain, in case you should ever need to recover the key or something.
The next time you run the same command (to renew the certificate) it won’t be asked.

So run the following command to generate the certificate:

$ sudo /opt/letsencrypt/letsencrypt-auto certonly -a webroot –agree-tos –renew-by-default \

–webroot-path=/home/user/webapps/appname \

-d website.com [-d sub.website.com] \

–e-mail=email@website.com

Then Generate Strong Diffie-Hellman Group
This may take a few minutes but when it’s done you will have a strong DH group at /etc/ssl/certs/dhparam.pem.
$ sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

Step 4: Configuring Nginx

After running the command that generates the certificates, you should have several files in /etc/letsencrypt/live/website.com/ (replace website.com by your own domain).
We are going to need just two of them for Nginx: fullchain.pem and privkey.pem.
Comment out or delete the lines that configure this server block to listen on port 80.
The beginning of your server block should look like this:

server {

server_name website.com www.website.com;

listen 443 ssl;

ssl_certificate /etc/letsencrypt/live/website.com/fullchain.pem;

ssl_certificate_key /etc/letsencrypt/live/website.com/privkey.pem;

# For Safari and iOS devices

ssl_session_cache shared:SSL:20m;

#Diffie-Hellman Group

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

ssl_prefer_server_ciphers on;

ssl_dhparam /etc/ssl/certs/dhparam.pem;

ssl_ciphers ‘ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA’;

ssl_session_timeout 1d;

ssl_stapling on;

ssl_stapling_verify on;

add_header Strict-Transport-Security max-age=15768000;

Lastly, outside of the original server block (that is listening on HTTPS, port 443), add this server block to redirect HTTP (port 80) to HTTPS.

server {
listen 80;
server_name website.com www.website.com;
return 301 https://$host$request_uri;
}

Put the changes into effect by restarting Nginx:

$ sudo service nginx restart

The Let’s Encrypt TLS/SSL certificate is now in place.

At this point, you should test that the TLS/SSL certificate works by visiting your domain via HTTPS in a web browser.
You can use the Qualys SSL Labs Report to see how your server configuration scores:

https://www.ssllabs.com/ssltest/analyze.html?d=website.com

Step 5: Automate the Certificate Renewal

Edit the crontab to create a new job that will run the renewal command every week.

$ sudo crontab –e

30 2 * * 1 /opt/letsencrypt/letsencrypt-auto renew >> /var/log/le-renew.log

35 2 * * 1 /etc/init.d/nginx reload

25 Feb

Migrating from Python 2.7 to Python 3.4

Last year, we made a decision to migrate to python 3.4 in 2016. This was not an easy decision as all our code base in developed using python 2.7 and it has served us very well.

Yesterday, We took the big step. This has already been delayed and was overdue.

First the Python 3.4 has to be installed, using the same process as in previous blog. However, replacing 2.7 with 3.4

the virtualenv has to be recreated for python 3.4.

pyvenv <virtual env>

thunder lock: disabled (you can enable it with --thunder-lock)
 uwsgi socket 0 bound to UNIX address xhod.sock fd 3
 Python version: 2.7.10 (default, Sep 21 2015, 12:21:01) [GCC 4.4.7 20120313 (Red Hat 4.4.7-16)]
 Set PythonHome to /home/uobis/webapps/xhod/venv
 ImportError: No module named site

I had to recompile uwsgi for python 3.4

/usr/local/bin/pip2.7 uninstall uWSGI

/usr/local/bin/pip3.4 install uWSGI

*** Starting uWSGI 2.0.12 (64bit) on [Thu Feb 25 22:31:10 2016] ***
compiled with version: 4.4.7 20120313 (Red Hat 4.4.7-16) on 25 February 2016 22:30:37
os: Linux-2.6.32-042stab108.8 #1 SMP Wed Jul 22 17:23:23 MSK 2015

01 Jun

uWSGI+NGINX on Linux (Centos 6)

Step 1 – Install NGINX
sudo yum install nginx
Step 2 – Add nginx to uobis group:
sudo usermod -a -G uobis nginx sudo chmod g+x /home && chmod g+x /home/uobis && chmod g+x home/uobis/webapps/ sudo -u nginx stat /home/uobis/webapps/

cat >> kabbu.ini
[uwsgi] module = wsgi master = true processes = 5 socket = kabbu.sock chmod-socket = 660 vacuum = true
Step 3 – The uWSGI Emperor – multi-app deployment
exec uwsgi --emperor /etc/uwsgi/vassals/ --master --logto /var/log/uwsgi.log
Step 4 – Create a Upstart File

To make sure stuff is automatically run, here’s a configuration file that needs to be placed in /etc/init/uwsgi.conf

description "uWSGI Emperor"

start on runlevel [2345] stop on runlevel [06] respawn exec uwsgi --emperor /etc/uwsgi/vassals/ --master --logto /var/log/uwsgi/emperor.log

Create the directories
mkdir /etc/uwsgi/vassals/
Then either reboot your system or:
sudo initctl start uwsgi sudo initctl reload-configuration
To check if its started
initctl list

Check the system log file or the emperor.log if uwsgi is not started

02 Oct

python+virtualenv on Cygwin

I have been trying to make my default environment for development to be

Install Python.
Download and unzip pip.
Install by going into the expanded directory and running python setup.py in a command prompt.
Set the %PYTHONHOME% system variable to the python base directory, (i.e. C:\Python27\) and adding the python base directory and script directory (i.e. C:\Python27\Scripts) to your %PATH% system variable.
Install Cygwin WITHOUT Python. The previous step tells Cygwin to use the Windows binary.
Install Cygwin-Virtualenvwrapper using pip install https://bitbucket.org/cliffxuan/virtualenvwrapper-for-cygwin-windows-python/get/tip.tar.gz
Install virtualenvwrapper-win using pip install virtualenvwrapper-win
Make a symlink between Cygwin’s virtualenvhome directory and Windows’s using ln -s /cygdrive/c/Users/<USER>/Envs/ ~/.virtualenvs
Add the following to Cygwin’s .bashrc file:
Go to C:\User\<username>\Env (or other %VIRTUALENV_HOME% location) and use virtualenv to start a new environment. Doing this allows virtualenvwrapper-win‘s workon command to work.

23 Jul

Deploying a Flask Project on WebFaction with Git [UPDATED]

I found it tedious having to redo the setup of my application on WebFaction whenever there is a new update of the application. I started looking for way to update the content of the application, without affecting the setup configurations of the application on WebFaction.

Step 1: Add your site to Your WebFaction Account

From your WebFaction console:

1. Log into your WebFaction Console

2. Go to your “Websites” tab, click the “Add New Website” button

3. Enter the website name and domains as needed for your project

4. Under the Contents section, choose Add an Application > Create a New Application

1. Name to “myapp”

2. Set App Category to “mod_wsgi”

3. App Type should be the most recent version of mod_wsgi that supports your Python version. Make sure that your Python version matches with your mod_wsgi choice ( “mod_wsgi 3.5/Python 2.7″ for this guide).

4. Click the Save button to add that application

5. Click the Save button to save your website

Now your WebFaction account should have a new domain, website, and mod_wsgi application.

Step 2: SSH into your new application

First, you’ll need to SSH into your webfaction account. Once you’re there, cd into your newly created application:

cd ~/webapps/myapp

In this directory, you’ll see two folders:

– apache2/ # This contains all of your Apache config and action files

– htdocs/ # This is the folder Apache to launch your project

Step 3: Upload your Flask project using Git

Upload the project/ folder to the application directory using Git. “myapp” directory is created

git clone https://username@bitbucket.org/uobis0/myapp.git

All following instructions will assume you’re still in this directory.

 virtualenv venv --python=python2.7

$ . venv/bin/activate
$ easy_install-2.7 flask    # Installs flask package for the app
$ pip install -r /myapp/app/requirements.txt  # Installs the packages for the app
$ deactivate

Now, my ~/webapps/myapp directory list looks like the following:

drwxr-xr-x 7 uobis uobis 4096 Nov 11 14:29 apache2/
drwxrwxr-x 5 uobis uobis 4096 Nov 11 14:40 venv/
drwxrwxr-x 6 uobis uobis 4096 Nov 14 16:46 myapp/
drwxr-xr-x 2 uobis uobis 4096 Nov 11 14:29 htdocs/
-rw-rw-r-- 1 uobis uobis 292 Nov 18 15:14 wsgi.py

Step 4: Edit apache2/conf/httpd.conf

Using your favorite command line editor, open up the apache2/conf/httpd.conf file:

vim ~/webapps/myapp/apache2/conf/httpd.conf

Load Alias module (optional)

You’ll see a section where Apache Modules are being loaded. I had to manually add the Alias module to the bottom of the list (shown below).

.
.
LoadModule wsgi_module       modules/mod_wsgi.so
LoadModule alias_module      modules/mod_alias.so    #Your version of mod_wsgi might not need to add this.

Modify Alias and <Directory>

Add the following parameters to your <Directory> section:

WSGIScriptAlias / /home/username/webapps/myapp/wsgi.py

<Directory /home/username/webapps/myapp/htdocs>
    AddHandler wsgi-script .py
    RewriteEngine on
    RewriteBase /
    WSGIScriptReloading On
</Directory>

Now for the final edit of the config file.

Step 5: Make sure your main file is right

Ensure that, if your structure is package, your project’s __init__.py file is launching your Flask application. This is what the file at ~/webapps/myapp/app/__init__.py should look like:

from flask import Flask

# Setting up the App
app = Flask(__name__)
app.config.from_object('config')

# Importing the views for the rest of our site
from app import views

if __name__ == '__main__':
    app.run()

If you are using modules, the files should contain the app instantiation

app = Flask(__name__)

This will work nicely with our wsgi.py file, which we’ll set up next.

Step 6: Modify the htdocs/wsgi.py file

WebFaction should have created this file. In it are a few scripts, but you can completely remove those. Here is what the ~/webapps/myapp/wsgi.py file should contain:

import sys

# Active your Virtual Environment, which I'm assuming you've already setup
activate_this='/home/username/webapps/myapp/venv/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this))

# Appending our Flask project files
sys.path.append('/home/username/webapps/myapp/myapp')

# Launching our app
from main import app as application

Step 7: Restart Apache

The last step will be to restart apache, like so:
~/webapps/myapp/apache2/bin/restart

Troubleshooting

If you’re having trouble, take a look at your logs in the ~/logs/users/ directory.

Application Updates

The steps above makes updates very easy:
cd ~/webapps/myapp/myapp
git pull https://username@bitbucket.org/accout/myapp.git

~/webapps/myapp/apache2/bin/restart

30 Nov

Deploying Flask on lighttpd

Deploying Flask seems a lonely taks, becuase of dearth of articles or blogs that explain in. This is even worse when deploying on a shared server.
Most people seems to treat a python deployment as a django deployment. Since, there are cheap and available Flask hsting site, i went with Django host, but confirmed that we can host any other framework.

The good thing is that, there too many similarities, and most packages are already installed on the host.

1. Open putty and log into the hosts server (s17.wservices.ch)
2. Check that the following following packages are installed

python and python-devel: the Python interpreter and its development package
lighttpd: The Lighty web server and its development package
install postgresql postgresql-contrib: The PostgreSQL database server and its development package
git: source code version control system (we will use it to download and update the application)
gcc: the C/C++ compiler (needed to compile Python extensions)
sudo: a tool that helps users run commands as other users.

If not, install them.

sudo apt-get python python-devel lighttpd httpd-devel mysql-server mysql-devel git gcc

3. Configure passwordless login (if needed)
4. Create a directory for the application, and install the application from BitBucket
mkdir app
cd app
git clone git://bitbucket.org/peppe/peppe-ng.git

5. Check User permissions
chmod -R 777 *

6. Setup the database
Go to https://panel.djangoeurope.com/databases/
Login with a username and password, and then create a database and dump the content of the dev db
pg_dump peppedb > ppdbdump.sql

Transfer the file to live host
sftp outfile.sql peppe@s17.wservices.ch

Load the Db on the live host
psql peppedb < ppdbdump.sql

7. Setup the webserver
Paste at the end of this file ~/lighttpd/lighttpd.conf

#Peppe.com
$HTTP[“host”] =~ “(^|.)peppe.com.ng$” {
fastcgi.server = (
“/flask.fcgi” => (
“main” => (
“socket” => env.HOME + “/mysite_project/mysite.sock”,
“check-local” => “disable”,
)
),
)
alias.url = (
“/media” => env.HOME + “/mysite_project/media”,
)

url.rewrite-once = (
“^(/media.*)$” => “$1”,
“^/favicon.ico$” => “/media/favicon.ico”,
“^(/.*)$” => “/flask.fcgi$1”,
)
}

Replace mydomain.com with the name of your domain. Be sure to escape all dots of your website’s name (put a backslash before it: .). Replace your_django_project/media with the path of you media directory (relative to your home directory). Replace mysite_project/mysite.sock with the path to your fastcgi socket file

Now you can launch your lighttpd:

~/init/lighttpd start

Whenever you make changes to the configuration, you can reload the configuration or restart lighttpd:

~/init/lighttpd reload

~/init/lighttpd restart

9. Configure the db and start
10. Install application updates