Delete git files from public GitHub history

/ chirale / Leave a comment

To delete git files uploaded accidentally to GitHub (or any other public repository) do these steps:

  1. Download https://rtyley.github.io/bfg-repo-cleaner/ as suggested by GitHub
  2. git clone –mirror GIT_REPOSITORY_URL
  3. cd path/to/cloned/repository
  4. Download BFG
  5. java -jar /path/to/download/dir/bfg-VERSION.jar –delete-files filename.ext
  6. Run the command specified by BFG (usually git reflog expire –expire=now –all && git gc –prune=now –aggressive)
  7. git push

If you get an error on pull, probably you haven’t cloned the repository as step 2.

Browsing the public history, any reference to the filename.ext file disappear.

Read more about BFG and the –mirror option on this discussion.

Advertisements

Mass delete old email on Gmail preserving Special and Tagged ones

/ chirale / Leave a comment

To mass delete old emails on Gmail type this search query in the search box of mail.google.com (or Gmail for Business):

after:2017/01/01 before:2017/31/12 -has:userlabels -is:starred

You can use these filters in any language but remember to use the YYYY/DD/MM format for the data (Year/Day/Month) for the after and before filters.

This search will show you all emails between January, 1st and December, 31st 2017 that:

  • Haven’t any User Label
  • Aren’t starred (without Star)

Change dates according to the time period you want to cover and select the select all checkbox inside the header to select all items from the Gmail dashboard.

Optionally, you can select them all using the dedicated link that appears after the step above.

These two criteria are usually enough to don’t delete important e-mails but you can add more exclusion criteria adding a minus sign before any new filter, e.g. unread. However, if you don’t use Stars and Labels you have to double-check email in the list before deletion to prevent to delete useful data.

This approach is very useful in these two scenarios:

  • To free space on the Gmail mailbox when it’s almost full.
  • To delete old emails to comply with regulations like GDPR at the end of their usable life.

Happy houseworks!

How to shrink a scanned PDF on Linux

/ chirale / Leave a comment

When you want to reduce the file size of a PDF document, this quick command using convert will shrink the original PDF file.

convert -density 150x150 -quality 60 -compress jpeg -colorspace Gray original.pdf new.pdf

This command is particularly useful against scanned documents, the jpeg quality will be 60% for 150dpi.

Converting an original 300dpi / color PDF to a 150dpi, greyscale PDF can reduce file size up to 50%. There will be some quality loss but in this way you can reduce file size enough to send scanned documents of dozens of pages via e-mail without using third-party services.

How to import .ovpn files on Ubuntu Linux network manager

/ chirale / Leave a comment

On Linux you don’t need to install OpenVPN because it’s already installed. However, configuration especially via the network manager can be tricky.

Install this additional package on your distro to display a new OpenVPN option in the network manager:

sudo apt-get install network-manager-openvpn-gnome

If you’re migrating from Windows and you’ve already a Windows installation of OpenVPN you can copy .key, .crt, .conf and .ovpn files from the OpenVPN location. Copy these files to your Linux home (e.g. ~/openvpn/) and reshape permissions to allow the access only to the owner.

After you’ve the .ovpn, .crt, .key files locally, you can test the connection using these commands:

cd ~/openvpn/;
sudo openvpn my-openvpn-file.ovpn

Type the sudo password, wait and the connection should be established successfully. Press Ctrl+C to stop the VPN from command line.

Now you can configure the Network Manager to accept the .ovpn file.

Click on the network icon on the top right corner of the screen, click current connection, select settings and look for VPN Settings from the opened window.

Click the + icon aside the VPN title and select Import from file…

Select the my-openvpn-file.ovpn you’ve checked before. A form containing user certificate, CA, private key and the gateway will be automatically filled. Input the password in the last field when needed.

It’s very important to select .ovpn and not .conf since the latter will not work.

If the private key is password protected you can also type the password and on Advanced you can do some fine tuning but it’s usually unnecessary.

On the Details tab, uncheck the automatic connection option if you don’t want to start the VPN at every login and choose if you want to allow other users to access the connection.

On IPv4 and IPv6 you can disable a specific protocol or limit the connection to “Use this connection only for resources on its network“. This last step is particularly important because using VPN can limit network connection.

Press Apply and you should be able to connect pressing the network icon on the top right corner > VPN > your VPN name.

To list saved connections:

 nmcli c

Programmatically connect / disconnect to VPN

If you need to write a script to use this imported connection, you can use openvpn command but you have to set all the parameters manually.

To reuse the saved connection instead, you can simply use nmcli to connect:

 nmcli con up id my-connection-name

And disconnect:

 nmcli con down id my-connection-name

 

Tested on Ubuntu 17 and 18.

 

PIL: ord() expected a character, but string of length 0 found (SOLVED)

/ chirale / Leave a comment

Using Django, and easy_thumbnails coupled with Pillow specifically I’m stumbled upon this error in PIL.ImageFile on PIL/_binary.py:

ord() expected a character, but string of length 0 found

This python error was so frequent I’ve done some research, coming up with nothing.

I’ve checked current Pillow version with:

pip freeze | grep Pillow

Getting:

Pillow==3.0.0

Then I’ve upgraded the Pillow package with this:

pip install Pillow --upgrade

The sofware was updated to the very last version (5.0.0) without any issue on easy_thumbnails backend or frontend.

Consequently, pip freeze returned:

Pillow==5.0.0

and errors are gone.

tl;dr: Update Pillow from 3.0.0 to latest version (5.0.0 by now)

Note: this error can be accompained by others:

  • unpack requires a string argument of length 2
  • string index out of range

Upgrading Pillow also correct these.

Failed to start Authorization manager (SOLVED)

/ chirale / Leave a comment

If you get the “Failed to start Authorization manager” while booting a Linux OS (e.g. CentOS) most likely you’ve a SELinux misconfiguration.

The main issue is that if you’ve restarted the system, you cannot login since OpenSSH service is not running. If you’ve a virtual server, even the console shipped by your provider is stuck on the Authorization manager error.

The reason behind this in my case was a completely wrong value in SELINUXTYPE.

The example below is valid for a Digital Ocean’s droplet running a CentOS 7 but can be adapted to other providers and distro.

To fix it I had to:

1. Boot the OS with initramfs (also called Recovery Kernel). You can do this on the dashboard provided by your virtual server provider (e.g. on Digital Ocean)

2. Open the console from your provider’s dashboard

3. Mount the root filesystem, e.g.

mkdir /mnt
mount /dev/vfsa1 /mnt

Where /dev/vfsa1 is your root partition.

4. Move broken configuration and recreate selinux config file manually with:

mv /mnt/etc/selinux/config /mnt/etc/selinux/config.BROKEN
touch /mnt/etc/selinux/config
cat > /mnt/etc/selinux/config

Then type these two lines:

SELINUX=disabled
SELINUXTYPE=targeted

Go to a new blank line after, then type Ctrl+D to quit. Since initramfs hasn’t a text editor like vi or nano, the simple cat > filename do the trick.

5. From your dashboard, change the kernel version to what you’ve used previously and Power Cycle (reboot) the machine using the dashboard because a reboot from initramfs in this state will restart initramfs itself.

Open the provider’s console on boot to see your system working again. Wait the services to start and then connect with your SSH client to see again your files and get again the control of your server.

Be careful when configuring SELinux again and create a shapshot and/or backup after you’ve restored your server…and before changing SELinux again.