Remove Docker overlays and images

To remove all the images and their overlays:

sudo docker rmi `docker images -aq`


If there are any remaining overlays in /var/lib/docker/overlay can we just delete these?

Try to prune docker:

 sudo docker system prune -a -f

Allowing docker write access to volumes mounted on NFS

My development machine (mydev) mounts several directories which are on an NFS mounted filesystem

When I run docker I wish to be able to access and write to the NFS filesystem.

The following is a brief technique:

Inside the dockerfile:


RUN useradd -m -d /home/mydocker mydocker; echo 'usermod -u $HOST_UID mydocker' >> ~/.bash_profile

# what order does this run in?
ENTRYPOINT source /root/.bash_profile && su - mydocker && /bin/bash


If I we to run docker with:

[me@mydev]$ sudo docker -i run -e"HOST_UID=`id -u`" -v /some/nfs/drive:/var/local/drive -t /bin/bash

Then mydocker will have the same uid as the user I ran the sudo with and should be able to access the NFS drive in the same was as the native user (me).

Git: decide that the changes I have made require a new branch

Okay, so I have cloned my repo and started working on a bug fix / feature.  When I come to wanting to commit the feature figure that I need to create  a feature branch.

So I can create and switch branch by:

git checkout [feature branch name]

git status # this will show the branch that my local sandbox is not pointing to)

git commit -m"my message" [my files]

git push origin [feature branch name]

Transfer svn repos to git

(based on the experience of following:  https://www.atlassian.com/git/tutorials/migrating-convert )

required: yum install wget git java git-svn

In the svn repos get the URL of the repository:

: svn info

Get the ids of all users who have committed work to the repository:

svn log --quiet | grep "^r" | awk '{print $3}' | sort | uniq




on the 'transfer machine' setup ssh access /access to your svn repos. For example if using ssh auth create a local user

adduser

then create ssh keys for that user:

ssh-keygen -t rsa

and copy the id_rsa.pub file onto the svn server and cat  >> authorised_keys file

su -

cd ~/home/

download the transfer script:

wget https://bitbucket.org/atlassian/svn-migration-scripts/downloads/svn-migration-scripts.jar




The migrations:

Test the you have the libs and if not install / upgrade

java -jar ~/svn-migration-scripts.jar verify

if all okay

generate an authors file. I hand crafted mine:
authors.txt

jb1 = Joe Bloggs < email@somewhere.com >

Then clone the repo (we used the standard trunk / branches/ tags layout):

git svn clone --stdlayout --authors-file=authors.txt svn+ssh://svn-server.somewhere.on.int.net/usr/src/local/svn-repos/web-world/ [your git repo name]

then see what there is totidy up:

cd [your git repo name ]

java -Dfile.encoding=utf-8 -jar ~/svn-migration-scripts.jar clean-git

java -Dfile.encoding=utf-8 -jar ~/svn-migration-scripts.jar clean-git --force

when happy create the repo on bitbucket (if not already)

prepare to send:

git remote add origin https://[git user]@bitbucket.org/[gitowner]/[your git repo name].git

git push -u origin --all

bish bash!

Notes:

For some reason your svn repo does not have tags or branches dirctories. The script will complain eg:

in [your git repo name]/.git/config

add to the section

[svn-remote "svn"]
    branches =

    tags =  

sails mocha: Running specific tests in mocha

To avoid running the entire test suite (when writing the tests), you can choose to only run specific tests based on a matching regular expresssion eg:

given snippet of packages file:

"scripts":{
    ...,
    "test": "PORT=9998 NODE_ENV=test mocha -R spec -b --recursive"
}

then

npm test --  --grep ManageSessions

will run any of the tests with a describe title matching *ManageSessions*

Windows docker running out of disk space

By default when installing Docker for windows the docker images file will be installed in the path configured in the Hyper-V manager, which for me happened to be in C:\Users\Public\Public Documents\Hyper-V.

To move the image file you need to:

1. Stop Docker.
2. Move the images file (MobyLinuxVM.xhdx) to the location you wish for Hyper-V to store the image
3. Open Hyper-v and in the setting change the default location to save this image
4. Uninstall and then re-install docker

voila!

locating the php.ini file

  phpinfo();
?>

Trouble running selenium script with Python Basic Auth

For pages protected by Basic Auth a little addition needs to be made to the Python script.

Whereas in a browser you can access the restricted pages by providing the username and password:

get http://username:password@somedomain.com/endpoint

you can not just put this in the url string of the Python webdriver class before you do you need to load a profile:

profile = webdriver.firefoxPorfile()

and call

profile.set_preference("network.http.phishy-userpass-length",255)

so that something like:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
profile = webdriver.FirefoxProfile()
profile.set_preference("network.http.phishy-userpass-length", 255)
driver = webdriver.Firefox(firefox_profile=profile)
driver.get("http://test0003:test@localhost/rasppi/")
driver.get("http://localhost/rasppi/")
print(driver.page_source.encode('utf-8'))
assert "Display" in driver.page_source
driver.close()

will work!

Create and manage Docker images on Windows and Docker Hub

To create a docker image on windows we need to install docker!

https://docs.docker.com/docker-for-windows/

and start the docker service.

Once this has been done prepare a directory structure, where we will organise our Dockerfiles for each image:

O:\sms67pc\docker\images\ 

Create a new directory containing a file named 'Dockerfile' for the image you wish to edit then cd to this directory edit the Docker file and once ready build the image:

# docker build -t [yourtag]  . 

to run the image interactively (cf login to the image) 

# docker run -i -t [yourtag] /bin/bash 

By logging in your can then test the further commands needed to build the image and add these to the docker file and repeat. Note, the images are not intended to be used as virtual machines and only one command can be run at the end (best to leave further commands to be run in the pipeline scripts; eg bitbucket pipeline will log you in 'interactively') 

You can start the Docker image exposing ports, to example, check the output of HTTP pages by mappng a port eg:

# docker run -i -i [yourtag] -p 8080:80 /bin/bash

If you were to start apache (inside the container) on port 80, then you should be able to view the web pages via http://localhost:8080/

Once happy upload to your docker hub account so that the images can be downloaded by tools like the bitbucket pipelines. 

# docker login 
# docker push [yourtag]