Tag Archives: python

Spellcheck web page by address

Looking for error in web pages can be very boring. I’ve tried some online tools but they seems too old, unsatisfying or premium only so I go to a 3 days programming marathon and I put online a brand new tool to do the job I need.

Go to Spellcheck This! and paste the URL address of the page you want to check and get the page checked for Deutsch, US English, Español, Français and Italiano.

The tool try to understand the language of the page and then return the result highlighting in different ways typing errors, names and acronyms on paragraphs, heading tags, tables and blockquotes.

spellcheck-result.jpg

It’s very far from perfect but it speeds up the spell check job. I’ve already spotted and fixed some minor errors on Wikisource and Wikipedia on very long texts.

Comment if you use and like it!

Programming note: Spellcheck This! is build with Python upon Flask and PyEnchant on the backend and Dust.js, jQuery and Foundation 6 Essential for the frontend.

Advertisements

Django and Drupal integration using drush via SSH

Some months ago I talked about how to achieve a unified login from Django to Drupal using drush. The basic assumption was that both Drupal and Django are on the same server. What if the two components are on different servers?

Paramiko is a SSH2 protocol library aimed to provide simple classes to make SSH connection. Let’s see how the code to call drush on command line changes.

Prerequisites:

  • paramiko
  • on your app settings.py add:
  • DRUPAL_SERVER_SSH_HOST     = '0.0.0.0' # Your host here
    DRUPAL_SERVER_SSH_USERNAME = 'YourRemoteServerUserHere'
    DRUPAL_SERVER_SSH_PASSWORD = 'YourRemoteServerPasswordHere'

    And then:

    assert request.user.drupal_id > 0
    # user id to log in
    drupal_id = str(request.user.drupal_id)
    output = ""
    try:
     # a list with command as first element and arguments following
     get_password_recovery_url = ["drush", "-r", settings.DRUPAL_SITE_PATH, "-l", settings.DRUPAL_SITE_NAME, "user-login", drupal_id]
     # via ssh http://stackoverflow.com/a/3586168/892951
     ssh = paramiko.SSHClient()
     # add to known_host the remote server key if it's not already stored
     # @see http://jessenoller.com/blog/2009/02/05/ssh-programming-with-paramiko-completely-different
     ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
     ssh.connect(settings.DRUPAL_SERVER_SSH_HOST, username=settings.DRUPAL_SERVER_SSH_USERNAME, password=settings.DRUPAL_SERVER_SSH_PASSWORD)
     ssh_stdin, output, ssh_stderr = ssh.exec_command(" ".join(get_password_recovery_url))
     output_lines = output.read().splitlines()
     # taking only the first line of the output:
     # e.g. 'http://example.com.it/user/reset/16/1369986816/67k7ReHi97FdtRfdrrXGqqesyz6FXyy7T8jqHiXxsrY/login'
    except:
     # @todo additional statements here
     pass
    finally:
     if ssh:
      ssh.close()
    
    if output_lines:
    drupal_login_url = output_lines[0].replace("http://example.com/", "http://%s/" % settings.DRUPAL_SITE_URL).strip()
    
    destination = "%s?destination=%s" % (drupal_login_url, settings.DRUPAL_LOGIN_DESTINATION)
     return redirect(destination)
    else:
     return HttpResponse('
    <h1>Wrong request</h1>
    ')
    

    This is the same code of the previous howto, with the difference that drush now is running on a different server of django. You can use the same method to do anything you have to with drush, any time you call this piece of code an SSH connection is opened.

    See also:

Create nice unicode PDF using Python

Today I started one of the less motivating activities in Python 2.x: encoding.

In Python 3 unicode will be everywhere, but as of the 2.6 version I’ve on one of the server I have to endure.

Objective: get data from a UTF-8 encoded json and print a nice PDF.

Tools: json, urllib2, fpdf, cgi

What you need:
pyfpdf: https://code.google.com/p/pyfpdf/downloads/list

  • Download fpdf-1.7.hg.zip or more recent
  • Unzip, enter the directory and python setup.py install
  • locate fpdf
  • cd /usr/lib/python2.6/site-packages/fpdf (or the directory name you got with locate)
  • Download unicode fonts for fpdf
  • Unzip and copy the fonts folder in the fpdf directory

Now you have a working FPDF with unicode support and unicode fonts. Start to write your script, I assume you’re using python 2.6, if not change python2.6 to your python version (e.g. 2.7) or remove version number in the heading (just python). As now FPDF works with Python 2.5 to 2.7.

Here I write a simple cgi-bin script, so you have to put it in the /var/www/cgi-bin directory (CentOS) or in /usr/lib/cgi-bin (Debian).

#!/usr/bin/env python2.6
#-*- coding: utf-8 -*-
from fpdf import FPDF
import json
import urllib2
import os
import cgi
import sys
# set system encoding to unicode
import sys
reload(sys)
sys.setdefaultencoding("utf-8")

Now get some arguments from url. These will be used to compile a query to a external json service.

# e.g. http://example.com/cgi-bin/myscript.py?lang=en&sid=2
sid = arguments.getlist('sid')[0]
lang = arguments.getlist('lang')[0]
# compile a request to get a particular element from an external json
dataurl = "http://example.com/external-json-source?lang=%s&sid=%s" % (lang, sid)
# load json from dataurl and convert into python elements
data = json.load(urllib2.urlopen(dataurl))
# the json has a user attribute: the user attribute has name and surname attributes as strings
user = data['user']
# title is a simple string
title = data['title']

Now you have to load the json from the external source. Json must be encoded in UTF-8:

lato_lungo = 297
lato_corto = 210
pdf = FPDF('L','mm','A4')
# add unicode font
pdf.add_font('DejaVu','','DejaVuSansCondensed.ttf',uni=True)
pdf.add_page()
pdf.cell(w=lato_lungo,h=9,txt=title,border=0,ln=1,align='L',fill=0)
pdf.set_font('DejaVu','',12)
# paragraphs rendered as MultiCell
# @see https://code.google.com/p/pyfpdf/wiki/MultiCell
# print key: values for each user['data'] dictionary attributes
for val in user.iteritems():
    pdf.multi_cell(w=0,h=5,txt="%s: %s" % val)
# finally print pdf
print pdf.output(dest='S')

Now:

  1. Open your browser and visit http://example.com/cgi-bin/myscript.py?lang=en&sid=2
  2. The external source http://example.com/external-json-source?lang=en&sid=2 is grabbed and converted into a python data structure. Both source and destination encoding are unicode utf-8.
  3. Data from external source are used to create the pdf.

You can use as many fonts as you have in the fpdf/font directory, just add those using pdf.add_font().

https://code.google.com/p/pyfpdf/downloads/list