Wednesday, 8 June 2016

Git on linux - a few fixes to some issues

A brief list of issues and ways to resolve when working on linux

Attempting to communicate with the server but this happens eg:

#git remote show origin

(gnome-ssh-askpass:18745): Gtk-WARNING **: cannot open display: localhost:12.0

Yep I do not have a display. Solution;


Attempt to push the first version of the repo but this happens:

#git push -u origin master

error: The requested URL returned error: 403 Forbidden while accessing

fatal: HTTP request failed

Turns out that I cut and pasted the code from github to setup the repo which included the line:

git remote add origin

This does not include the username I wish to login as I therefor eneed to change this locally to be:

To do this:

# git remote set-url origin

And away we go:

I did a #git add * but too much got added including my config/local.js file. 
Before the commit remove this:

# git reset config/local.js

Including emoji / unicode characters to email subject from a scipt

💣 Time to add special characters to your email subjects

I came across this problem when a client asked for emails from my web application to stand out in their inbox.  The emails were informing a user that an alarm had gone off and they would like a bomb or similar to appear in their inbox. 

After a bit of googling and finding many sites that explained how to do this manually (cut and paste characters or using particular 'clipboard' style tools).  I figured out the answer of how to embed the unicode into the message.

So given my script (a nodejs application) I followed the recipe below:

Find the character the user is interested in for example search a site like

Once you have found the required character locate the UTF-8 (hex string) in the case of the alarm clock ( 'e28fb0'

Embed this in the subject string via: =?UTF-8?Q?=E2=8F=B0?= (ie enclose the hex in =?UTF-8?Q?= and ?=).

Send message and as long as their mail agent supports the character hey hoe!

Monday, 21 September 2015

Sails waterline ORM on an MySQL view

I am required to keep a track of the number of selections a user has made so that when they are below a particular number they can be contacted.

To do this I need to monitor a group by with having count(*) query, for which I decided to generate a view:

create view progress_view as select user.username, display_name, email, year_of_entry, count(*) as num_choices from user left join selection on user.username = selection.username where progress = 1 group by username;

unfortunately defining a regular sails model object fails to load as it complains that we are 'Trying to define a collection (progress_view) which already exists.'

to get around this and to prevent a regular query attempting to retrieve the default fields createdAt, updateAt, id when using the .find() query on the table the following was wadded to the model definition:

var Progress = {
  // Enforce model schema in the case of schemaless databases
  schema: true,

  autoPK: false,
  autoCreatedAt: false,
  autoUpdatedAt: false,
  tableName: 'progress_view',
  migrate: 'safe',

  attributes: {
    username  : { type: 'string', unique: true },
    email     : { type: 'email' }, //,  unique: true },
    display_name     : { type: 'string' }, //,  unique: true },
year_of_entry : { type: 'integer' }, //only on some will be added -> or maybe all, but generally in the inst_name?
num_choices : { type: 'integer' }, //note the minimum here is always 1! evern if the student has not chosen on (as we are identifying with left join)

module.exports = ProgressProgress;

hurrah - we can query our view!

Thursday, 17 July 2014

Copying google calendars.

For static calendars (one off consolidation):



Wednesday, 5 March 2014

Docker Notes - setting up a lisp container

These notes are to help serve as a reminder for how I setup / played with Docker to host a lisp environment client.

In a new directory create a file called 'Dockerfile' and populate as follows:

FROM centos
MAINTAINER Stephen Shorrock
# Install sbcl
RUN rpm -Uvh
RUN yum install -y sbcl

Then in that same directory build the docker image/container:

docker build -t="sbcl" ./

Once built run the container as a new bash shell:

docker run -i -t sbcl /bin/bash

start lisp:


or run the sbcl interpreter striaght away:

docker run -i -t sbcl sbcl

You could even run a lisp script on the local drive by mounting a path, eg if your script was /tmp/myscripts/script1.lisp

eg (content of /tmp/myscripts/script1.lisp)

(write-line "Hello, World!")

docker run -i -v /tmp/myscripts:lisp_scripts -t sbcl sbcl --script lisp_scripts/script1.lisp

#: Hello, World!


once finished exit lisp


and quit the docker container:

Ctrl-P Ctrl-Q

To attach back into the container, find the id of the running container

sudo docker ps

Then attach back:

sudo docker attach

Friday, 1 June 2012

Raspberry Pi: automatically mount usb as hard drive

I'd like to configure my raspberry PI to auto configure itself on the USB memory stick that I boot it up with. Its USB memory stick will essentially be the 'hard disk' for the device storing various applications and dependant on which stick is found the raspberry will load and run in a particular way.

 Stage 1 Finding the memory stick and mounting

All attached usb devices can be listed by:

lsusb or lsusb -v.

To mount we need to be root.  For my image ( to change to root I need to issue:

sudo su -

After you have plugged the USB key in there are several ways in which you can get the device id (eg /dev/sda1), the most common is probably via dmesg.  However if you are running udev (which my release of Rasberry Pi is) you can reference the device by several aliases id, label, path, uuid you can list the aliases under each by: ls /dev/disk/by-

To allow copying of the hard disk I intent to label the usbkey so to replace the key I can label a new one with the same label.

My list of currently inserted keys is found by:

ls /dev/disk/by-label


The first thing I will do is change the label of my disk from USB2 to something else: 'PI1'

to do this I find the true device ID for my disk:

ls -l /dev/disk/by-label/PI_1
>lrwxrwxrwx 1 root root 10 Apr 17 14:15 /dev/disk/by-label/PI_1 -> ../../sda1

ie the device label is a symbolic link (shown by the '->') to two directories above '../../' named sda1

I issue the command mlabel -i /dev/sda1 ::PI1 to change my label

now I can add a mount command in the /etc/fstab file:

/dev/disk/by-label/PI1  /mnt/usbkey vfat defaults 0 0

If I were to want another PI version I could follow the procedure as above (using a different label eg PI2).  Then also add this to the /etc/fstabs file to mount in the same place:

/dev/disk/by-label/PI2 /mnt/usbkey vfat defaults 0 0

Whichever key inserted will be mounted to the /mnt/usbkey directory

Monday, 30 April 2012

WTForms - validating values against other others

Validating form values against each other

Again very basic - I wish to make sure that the value of a field entered is greater than another field. To do this I needed to create a custom validator. This custom validator was based on the EqualTo validator. I called it's class GreaterThan.

I could edit the wtforms/ file and add a new validator in there how ever this would not make library upgrades easy. Instead I created a mywtforms directory within which I created a file which when

 import mywtforms 

is used will load the code in this file. The contents of the file look like:

#need help figuring out what to import here I've creating so methods that inherit from wtforms and its structure
#TODO re-read :
from wtforms import validators, widgets
from wtforms.widgets import *
from wtforms.fields import *
from wtforms.form import Form
from wtforms.validators import ValidationError

class GreaterThan(object):
    Compares the value of two fields the value of self is to be greater than the supplied field.

    :param fieldname:
        The name of the other field to compare to.
    :param message:
        Error message to raise in case of a validation error. Can be
        interpolated with `%(other_label)s` and `%(other_name)s` to provide a
        more helpful error.
    def __init__(self, fieldname, message=None):
        self.fieldname = fieldname
        self.message = message

    def __call__(self, form, field):
            other = form[self.fieldname]
        except KeyError:
            raise ValidationError(field.gettext(u"Invalid field name '%s'.") % self.fieldname)
        if != '' and <
            d = {
                'other_label': hasattr(other, 'label') and other.label.text or self.fieldname,
                'other_name': self.fieldname
            if self.message is None:
                self.message = field.gettext(u'Field must be greater than %(other_name)s.')

            raise ValidationError(self.message % d)

Later I plan to split up and place the code into directories mirroring those in the wtforms directory

To use the validator I could create a form such as:

import re

def filter_field(value):
   return re.sub('\d{2}$','00',value)

class MyForm(Form):
   field_one = TextField('Field 1',validators=[validators.Regexp('^[0-9]*)',
                             message='Must be an integer')],
                             filters=[lambda x: x*100, filter_field])
   field_two = TextField('Field 1',validators=[validators.Regexp('^[0-9]*)',
                             message='Must be an integer'),GreaterThann('field_one','field two must be greater than field one')],
                             filters=[lambda x: x*100, filter_field])