How to shrink a scanned PDF on Linux

When you want to reduce the file size of a PDF document, this quick command using convert will shrink the original PDF file.

convert -density 150x150 -quality 60 -compress jpeg -colorspace Gray original.pdf new.pdf

This command is particularly useful against scanned documents, the jpeg quality will be 60% for 150dpi.

Converting an original 300dpi / color PDF to a 150dpi, greyscale PDF can reduce file size up to 50%. There will be some quality loss but in this way you can reduce file size enough to send scanned documents of dozens of pages via e-mail without using third-party services.

How to import .ovpn files on Ubuntu Linux network manager

On Linux you don’t need to install OpenVPN because it’s already installed. However, configuration especially via the network manager can be tricky.

Install this additional package on your distro to display a new OpenVPN option in the network manager:

sudo apt-get install network-manager-openvpn-gnome

If you’re migrating from Windows and you’ve already a Windows installation of OpenVPN you can copy .key, .crt, .conf and .ovpn files from the OpenVPN location. Copy these files to your Linux home (e.g. ~/openvpn/) and reshape permissions to allow the access only to the owner.

After you’ve the .ovpn, .crt, .key files locally, you can test the connection using these commands:

cd ~/openvpn/;
sudo openvpn my-openvpn-file.ovpn

Type the sudo password, wait and the connection should be established successfully. Press Ctrl+C to stop the VPN from command line.

Now you can configure the Network Manager to accept the .ovpn file.

Click on the network icon on the top right corner of the screen, click current connection, select settings and look for VPN Settings from the opened window.

Click the + icon aside the VPN title and select Import from file…

Select the my-openvpn-file.ovpn you’ve checked before. A form containing user certificate, CA, private key and the gateway will be automatically filled. Input the password in the last field when needed.

It’s very important to select .ovpn and not .conf since the latter will not work.

If the private key is password protected you can also type the password and on Advanced you can do some fine tuning but it’s usually unnecessary.

(Only for Ubuntu < 20) On the Details tab, uncheck the automatic connection option if you don’t want to start the VPN at every login.

You can also choose if you want to allow other users to access the connection after save.

On IPv4 and IPv6 you can disable a specific protocol or limit the connection to “Use this connection only for resources on its network“. This last step is particularly important because using VPN can limit network connection.

Press Apply and you should be able to connect pressing the network icon on the top right corner > VPN > your VPN name.

To list saved connections:

 nmcli c 

Programmatically connect / disconnect to VPN

If you need to write a script to use this imported connection, you can use openvpn command but you have to set all the parameters manually.

To reuse the saved connection instead, you can simply use nmcli to connect:

 nmcli con up id my-connection-name 

And disconnect:

 nmcli con down id my-connection-name 

 

Tested on Ubuntu 20, 18 and 17.

 

PIL: ord() expected a character, but string of length 0 found (SOLVED)

Using Django, and easy_thumbnails coupled with Pillow specifically I’m stumbled upon this error in PIL.ImageFile on PIL/_binary.py:

ord() expected a character, but string of length 0 found

This python error was so frequent I’ve done some research, coming up with nothing.

I’ve checked current Pillow version with:

pip freeze | grep Pillow

Getting:

Pillow==3.0.0

Then I’ve upgraded the Pillow package with this:

pip install Pillow --upgrade

The sofware was updated to the very last version (5.0.0) without any issue on easy_thumbnails backend or frontend.

Consequently, pip freeze returned:

Pillow==5.0.0

and errors are gone.

tl;dr: Update Pillow from 3.0.0 to latest version (5.0.0 by now)

Note: this error can be accompained by others:

  • unpack requires a string argument of length 2
  • string index out of range

Upgrading Pillow also correct these.

Failed to start Authorization manager (SOLVED)

If you get the “Failed to start Authorization manager” while booting a Linux OS (e.g. CentOS) most likely you’ve a SELinux misconfiguration.

The main issue is that if you’ve restarted the system, you cannot login since OpenSSH service is not running. If you’ve a virtual server, even the console shipped by your provider is stuck on the Authorization manager error.

The reason behind this in my case was a completely wrong value in SELINUXTYPE.

The example below is valid for a Digital Ocean’s droplet running a CentOS 7 but can be adapted to other providers and distro.

To fix it I had to:

1. Boot the OS with initramfs (also called Recovery Kernel). You can do this on the dashboard provided by your virtual server provider (e.g. on Digital Ocean)

2. Open the console from your provider’s dashboard

3. Mount the root filesystem, e.g.

mkdir /mnt
mount /dev/vfsa1 /mnt

Where /dev/vfsa1 is your root partition.

4. Move broken configuration and recreate selinux config file manually with:

mv /mnt/etc/selinux/config /mnt/etc/selinux/config.BROKEN
touch /mnt/etc/selinux/config
cat > /mnt/etc/selinux/config

Then type these two lines:

SELINUX=disabled
SELINUXTYPE=targeted

Go to a new blank line after, then type Ctrl+D to quit. Since initramfs hasn’t a text editor like vi or nano, the simple cat > filename do the trick.

5. From your dashboard, change the kernel version to what you’ve used previously and Power Cycle (reboot) the machine using the dashboard because a reboot from initramfs in this state will restart initramfs itself.

Open the provider’s console on boot to see your system working again. Wait the services to start and then connect with your SSH client to see again your files and get again the control of your server.

Be careful when configuring SELinux again and create a shapshot and/or backup after you’ve restored your server…and before changing SELinux again.

HTTPS: how to add TLS ciphers on nginx (update regularly)

HTTPS is a great improvement to a website security. However, HTTPS comes in different flavours and among these there are very weak ones.

Among protocols, SSL have to be avoided because it is not secure. Its successor, TLS, comes in different versions and supports different ciphers. To be short, the cipher is the encryption method/algorihms the website and the client use to talk each other.

The combination of protocols and ciphers available to implement HTTPS will limits the type of clients capable to access the website.

To be sure your website will not lose traffic, you have to balance the strongest ciphers available with the most compatible but still secure, dropping all weaker ciphers.

Check the strenght of your HTTPS implementation

If you’ve already implemented HTTPS on your website, first you’ve to check ist current security status of protocols and ciphers.

Check your hostname on Qualys SSL Labs pasting the HTTPS protected domain on the Test your server section. It’s a fast method with a very detailed output for public websites.

The report will give your hostname a rank, a detailed list of issues, browser support, and the complete list of supported ciphers. Among these ciphers, you can get some ciphers highlighted in yellow. You have to get the rid of these no matter what.

The list of ciphers actually differs from a typical cipher declaration on nginx because nginx can use the OpenSSL naming and Qualys uses IANA naming.

Here’s an helpful conversion table by Mozilla where you can convert IANA to OpenSSL and the other way round. Take note of the weak ciphers but wait before start to cut your cipher declaration on nginx.

You’ve to check how many visitors you’ll lose after the cut first.

Get the website statistics

Using Google Analytics or similar services and software, go to the Audience > Technology > Browser to get a list of your visitors’ browsers. Select a timespan like the last year or less.

You can add Browser version or OS version as secondary dimension to match the list of supported browsers from SSL Labs. You’ll get something similar:

analytics-technology-browsers

Well, someone is still using Internet Explorer 9.0 in 2018.

Since Internet Explorer running on old Windows versions (like XP) is one of the most troublesome combination, check how many visitors use this legacy software.

On Google Analytics type on the search box “Internet Explorer” and you’ll get the browser usage of this legacy browser. Select OS version as secondary dimension to get a list of OSes using IE.

Compare this list with the report from SSL Labs and with the conversion table from Mozilla cited above and count the number of visitors you want to cut off from your website in the sake of security.

Cut the weak ciphers

Trimming down the ciphers declaration on nginx conf you’ll get something like this:

ssl_ciphers ‘ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!3DES’;

Each cipher is separated by a ‘:’ and at the end some elements (typically using OpenSSL naming) are forbidden with a ‘!’.

Here’s the context:

server {
        # the port your site will be served on
        listen      443 ssl;
        # the domain name it will serve for
        # substitute your machine's IP address or FQDN
        server_name example.com www.example.com;
        ssl_certificate /path/to/fullchain.pem;
        ssl_certificate_key /path/to/privkey.pem;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_prefer_server_ciphers on;
        ssl_dhparam /etc/ssl/certs/dhparam.pem;
        # cfr. ........................................
        ssl_ciphers ** PASTE CIPHERS HERE **;
        ssl_session_timeout 1d;
        ssl_session_cache shared:SSL:50m;
        ssl_stapling on;
        ssl_stapling_verify on;
        add_header Strict-Transport-Security max-age=15768000;
        charset     utf-8;
        # This is for Let's Encrypt
        location ^~ /.well-known {
                alias /path/to/.well-known;
                allow all;
        }

        # max upload size
        client_max_body_size 75M;   # adjust to taste

        location /webpath  {
                alias /path/to/web;
        }
}

Change the conf file, reload nginx (on CentOS 7 systemctl reload nginx) and then re-run the SSL Labs test.

The Qualys’ tool will show you the new incompatibility with legacy browsers in the Handshake Simulation section:

tls-handshake

Modern protocols and ciphers implemented using the above declaration on nginx cut off IE 8 on XP and IE 6, the report explain.

According to the technology used by visitors of the analyzed website, few visits are sacrificed for better security for both visitors and host.

Tune these settings according to your needs, keep monitoring the tecnology used by site visitors and dropping legacy system progressively, with Modern compatibility as a (not so) long term objective.

Certbot: auto-renew LetsEncrypt certificate on cron

Certificates renewal can be difficult to automate leading to errors that will mark the website as “Insecure”.

Before continue, ensure you haven’t the /etc/cron.d/certbot file launching:

less /etc/cron.d/certbot

If a crontab appear, you already have an automatic renew enabled via a certbot plugin like nginx or apache (the preferred method) and you shouldn’t do nothing.

The following will apply to standalone method, a small webserver provided by Let’s Encrypt that will listen the 80 port, without any existing auto-renew cron script. It’s very useful in cases where a plugin for your webserver is not available, like haproxy.

In this case, here’s how to automate certificate renewal with nginx as webserver:

sudo nano /etc/cron.d/certbot-custom

And then add to the crontab these lines pressing A to edit:

37 02 * * * root certbot -q renew --pre-hook="systemctl stop nginx" --post-hook="systemctl start nginx"

The root before certbot is the user who will run the command.

Every day at 02.37 the certificate will be asked for renewal in quiet mode to avoid to spam on root mailbox, especially useful if you use external SMTP server for system mails on Linux.

After you’ve typed these lines, type:

:wq

To write and quit. The /etc/cron.d/certbot-custom file will be easily accessible and maintainable compared to crontab -e, but it is a valid method too. If you use crontab -e, do it as root and do not add root after the last *.

To exactly mimic the certbot cron plugin, transform the line on /etc/cron.d/certbot-custom file to:

37 02 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system &amp;&amp; perl -e 'sleep int(rand(43200))' &amp;&amp; certbot -q renew --pre-hook="systemctl stop nginx" --post-hook="systemctl start nginx"

Before run, all paths will be checked, and a random wait will be added. If your webserver requires additional commands, create a script to be executed in –post-hook.

Other Linux distributions

If you’re using a different Linux OS you can locate certbot-auto using the following command:

whereis certbot

And then you can use it on the crontab.

You’ve also to use the alternative of systemctl for your system to refresh the certificates on the webserver.

If you’ve to do get your first certificate, here you can get more information about how to install free Let’s Encrypt certificates on nginx.

When auto-renew certificates

Since certificates lasts for about three months, you can tell crontab to run this every week instead every day accordingly.

Adding the 0 on the 5th position will tell crontab to run the command every Sunday at 2 AM (2nd position) and 37 minutes (1st position).

If you find this syntax difficult you can use crontab.guru to easily generate the crontab.

Updated on august 16th, 2019

A new language for Social Media managers on Twitter

Last months I looked for a tool to shape my community on Twitter to follow interesting profiles and to increase my followers.

I had bad experiences using integration from third party (app) so I wanted this tool to be able to create my own app on Twitter without 3rd party involvment for better security and privacy.

Since I wanted real new followers and I don’t want to violate Twitter policies I looked for a tool able to select and filter users from my network, choosing only those whom are relevant to follow.

I wasn’t looking for a new, fancy SaaS website with a monthly fee to pay, and I want to handle many accounts at the same time without additional costs.

I wanted a tool able to run on my own PC and with the full access to the source code to avoid my data to be stolen and to understand its inner mechanics.

Well, that tool doesn’t exists at the time.

So I decided to write my own.

The fancy app

When I start to develop a full-featured Twitter app for desktop with a user interface running on my local machine. I started to add icons for the actions, tables to list users, buttons to do actions, a lot of checkboxes to select users and so on.

The result was a good-looking app that works. But when I tried to filter and select users using some criteria it all became clumsy.

As a programmer, I started to feel my own application as a cage.

I wasn’t allowed to search for users on my network for multiple or complex criteria. I wasn’t able to merge, diff or intersect different set of users.

Then I realized that what I was really looking for wasn’t an app but a brand new scripting language to manage social networks.

The programming language

I started to look for open source solutions able to create this new programming language for social media management using Python 3. There is a small bunch of instructions to add to this programming language but I want it to be efficient and well-designed.

Here comes in help TextX by the professor Igor Dejanović, a parser build on top of the Arpeggio PEG. Among all formal languages and parsers, Parsing Expression Language (PEG) seems to me the better for my purpose and the most modern approach to parsers.

The grammar of this new language is written on a single file and can be graphically represented using DOT language. Then TextX use the grammar to parse the language using a Meta-model where the language comes to life.

.ows language grammar

.ows language grammar

To handle the Twitter management I used the solid Tweepy by Joshua Roesslein and to query the social network SQLAlchemy and SQLite.

Scripts are launched by command line and an interactive console with history is available to manage your Twitter account using the scripting language using the Python Prompt Toolkit by Jonathan Slenders.

JournaKit Followship .ows

All these free software / open source tools among others are the construction blocks used to create this bare-bones social media managers’ tool for Twitter.

A simple scripting language to manage and expand your network of followers and friends with complex queries, running on my PC, registered with custom application on Twitter and executable on many accounts at the same time.

Its name is JournaKit Followship .ows and it’s available on Gumroad. The complete application, the source code and a comprehensive user manual are provided allowing you to master the .ows language.

This is the first application of the JournaKit suite aiming to help journalists and writers whom use the web to share their works and to discover new sources and contacts.

Comment this article or contact me privately if you want to know more about it.

10 essentials steps to use Elementary OS as media station

I was looking for an easy and good alternative for Windows 7 on a laptop used as a media station to watch Netflix and Crunchyroll and I choose Elementary OS.

This GNU/Linux distribution is based on Ubuntu and it’s relatively easy to install. Anyway, it needs some steps to work great as media station. Here we go:

  1. Make AppCenter Work
    1. Click on Applications on the top left corner of the screen
    2. Click on Terminal
    3. On the console type these commands:
      sudo dpkg --configure -a
      sudo apt update
      
  2. Update your system
    1. Open the AppCenter moving the mouse at the bottom on the screen, then click on the store icon
    2. Click on Updates on the top of the window
    3. Click on Update All and wait until all packages are installed
    4. When finished, click on the Power icon in top right corner of the screen and restart
  3. Update language (if different than English)
    1. Go to the System settings on the bottom bar
    2. Choose Language & Region
    3. A yellow box should appear telling the language installation is not complete: click on Complete installation
    4. Type the user password you’ve choosen during the installation
    5. Click on Unlock
    6. Select your language, region and format
    7. Log out using the top right power button and log in again. The OS is now translated.
    8. Click on Update names to change home directories names according to the selected language (or keep it in the English version)
  4. Install VLC
    1. Go to AppCenter
    2. On the top right corner of the AppCenter search “vlc”
    3. Click Install
  5. Install your preferred browser
    1. On the top right corner of the AppCenter look for your favourite browser:
      1. If you’re looking for Firefox, type “firefox” and install it
      2. If you’re looking for Chrome, type “chromium” and install it. It’s the open source alternative of Chrome.
  6. Make your browsers ready to watch streaming shows:
    1. If you’re using Firefox, go to Menu > Preferences > Content > and flag Play DRM content checkbox. It will allow services using this meh technology.
    2. Optional: Install Flash Player for Linux if needed (automatically installed in Firefox):
      https://get.adobe.com/it/flashplayer/otherversions/
  7. Optional: Translate your browser
    1. On Firefox, type about:addons in the address bar
    2. Search for your language and install
      1. Language pack
      2. Dictionary
  8. Optional: Install Office Productivity Tools:
    1. On AppCenter, search for LibreOffice, LibreOffice Writer and LibreOffice calc and install them (one by one) to open Word and Excel files or Open Office files.
  9. Optional: get new wallpapers automatically
    1. In the AppCenter search for “Variety
    2. Install and configure it
    3. You can add nice quotes and a clock to the desktop editing Preferences
  10. Switch the sound to the TV when the HDMI cable is plugged into the port:
    1. By default, when you plug the HDMI cable to the PC you can hear the sound coming from PC speakers
    2. To solve this issue open the Terminal and type
      sudo bash
      

      And type your password.

    3. Follow this howto to automatically redirect sound to the TV when it’s plugged in. To create or edit files you can use:
      nano /path/to/file
      

With these steps your brand new media station is ready for watching streaming relying on the strong security implemented in GNU/Linux systems.

10 steps to install LyX on Windows

Here the steps to install LyX on Windows:

  1. Download MiKTeX to provide LaTeX (usually the 64bit version)
  2. Select the installation directory and remember the path
    mikitex
  3. Complete the installation procedure
  4. Download and install LyXWinInstaller
  5. Press Next until a dictionary list is presented: add dictionaries for your languages
  6. Before installing the MiKTeX will be shown:
    lyxwin
  7. Click Install and proceed.
  8. You should get an Update popup for MiKTeX, check the connection settings and then proceed and click Next to update all packages
  9. After the update close the MiKTeX popup and wait to LyX installation to finish: it should take minutes, to see progress click “See details” so you can see it’s not hanged up but it’s simply fetching different templates you can use later.
  10. Select Launch LyX and start using LyX

Now you can enjoy LyX for Windows to create PDF using LaTeX without having to deal with the usual formatting hassles of a WYSIWYG editor.

If you’re a beginner or you want to jog your memory about LyX select Help > Tutorial.

To export to PDF you can user File > Export and then select PDF LuaTeX or PDF XeTeX. In my case LuaTeX generate larger document than XeTeX but both are nicer than those generated by ps2pdf.

Note: If you get some error by MiKTeX regarding SOMETHING.sty, close LyX and in the Windows application finder search MiKTeX Package Manager and run it as Admin. Then look for SOMETHING and install it. This will fix the issue!

Linux: MySQLdb on virtualenv with –no-site-packages

In the past it was difficult to get MySQL working on virtualenv without using system packages. Now you can have a real separated environment with simple steps:

  1. Follow this guide to install virtualenv using this command:
    virtualenv myproject --no-site-packages

    This command will install a new virtualenv inside a new directory myproject created by the command itself.

  2. Activate virtualenv:
    source myproject/bin/activate
  3. Upgrade setuptools
    pip install pip --upgrade
  4. You can now install MySQLdb, inside the package MySQL-python:
    pip install MySQL-python
  5. Now do a simple test trying to connect to an existing database:
    python
    import MySQLdb
    db = MySQLdb.connect(host="localhost",   # your host, usually localhost
                         user="chirale",         # your username
                         passwd="ITSASECRET",    # your password
                         db="chiraledb")         # name of the database
    cursor = conn.cursor()
    cursor.execute("SELECT VERSION()")
    row = cursor.fetchone()
    print "server version:", row[0]
    cursor.close()
    conn.close()
    

Tested on CentOS 7, Python 2.7

Tip: If you are starting to create a database doing all the dirty work alone you’ve to give SQLAlchemy a try. You can use like an ORM or a lower level as you wish.

See also

The Hitchhiker’s Guide to Python
Simple MySQLdb connection tutorial

About the same topic

Python: MySQLdb on Windows virtualenv (w. figures)