Machine Learning to Love

Data Scientist, Developer, Lover of Computer Vision and Machine Learning. Technically a Neuroscientist.

Remote Development in Visual Studio using SSH — January 27, 2021

Remote Development in Visual Studio using SSH

I really, really, really hate using graphical interfaces through any sort of remote desktop solution (unless we’re talking about Stadia which is the bee’s knees). I don’t like laggy cursors, slow-to-load windows, and the fact that IDEs will often break when you try to open them remotely on Ubuntu. So, we have ssh.

To test it through Once you’re on the same VPN as the machine you’re logging into, you can open up a terminal on your local machine, and type:

ssh your-user-name@host-ip

where your user name is what you use to log into the host, and the host-ip is the local IP address of the machine. This will look something like jackfrost@10.1.202.193. (SSH works natively in Ubuntu, but you might need to configure ssh on your local machine. Or, just skip to the next step). If this works, it’ll ask you for a password, and voila, you’re logged into the remote machine through the terminal. Now close the terminal and open Visual Studio.

This part is super simple. All you need to do is download and install this extension for VScode. Once you do, open the command palette, (Ctrl/Command+Shift+P), and type in “ssh” (without the quotes). Select “Connect to Host” and type the same username@host-ip combo you did earlier.

Now, you’re running VScode on your local machine, but accessing the files and terminal from your host! YAY!

Backup and a dockerized postgres database — October 20, 2020

Backup and a dockerized postgres database

This is an awkward step so I figured I’d add a note on how to get it done… I assume this is a pretty dumb way of doing it but hey, works for now.

To back up the databases:

docker exec -t db_container_name pd_dumpall -c -U postgres > dump`date +%d-%m-%Y"_"%H_%M_%S`.sql

and to restore from the backup:

cat your_dump.sql | docker exec -i db_container_name psql -U your_username

Some GitHub tips — October 15, 2020

Some GitHub tips

Alrighty, so here’s a running list of things I use most often with Git and GitHub:

To download a new repository into a folder that’s automatically created:

git clone https://github.com/mikeusru/repo_name.git

To update the current repository with changes from github:

git pull

When you make some changes that you want to save (a few times a day, or every time you start something new):

git add .
git commit -m "a summary of the changes"
git push

When you made some changes locally but want to download whatever other changes have been made in the github, and without losing the local changes:

git stash
git pull
git stash apply

When you want to make a new branch

git checkout -b new_branch_name
git push --set-upstream origin new_branch_name

when you want to switch to a remote branch

git fetch
git checkout -b new_branch_name origin/new_branch_name

when you’re done with a branch and want to merge it into the main branch

git checkout old_branch_name 
#main or master, whatever the main branch is called

git merge new_branch_name

Enter a running docker container — October 14, 2020

Enter a running docker container

When there’s a running docker container and you need to execute some code in it through the terminal, you need to enter it first.

1. To enter the container, get its name first. in the terminal type:

docker ps

and you’ll get an output of the currently running containers, including some stupid name like fastidious_fruitfly or jungle_beaver. That’s the name of the container you want to enter.

2. Enter the container with the following code (substitute in whatever your container’s name is):

docker exec -ti fastidious_fruitfly /bin/bash

3. And voila! you’re now inside the docker container’s ecosystem, able to run bash commands.

Downloading a sharepoint file through command line — October 9, 2020

Downloading a sharepoint file through command line

Since Microsoft doesn’t give an obvious way to download sharepoint files through their links, this needed a bit of research. The trick is to share the file (where anyone with the link can download it) and change the link manually.

At the end of the link there’s a question mark. Replace the text after the question mark to download=1

For example,
https://my.sharepoint.com/:u:/g/XXX/XXXX-bunchofRandomText?e=kRlVi

gets replaced to

https://my.sharepoint.com/:u:/g/XXX/XXXX-bunchofRandomText?download=1

Now you can use wget or curl on the link. The problem is that the filename may be wrong, so you have to specify it yourself with -O. for example,

wget https://my.sharepoint.com/:u:/g/XXX/XXXX-bunchofRandomText?download=1 -O /path/to/new/file.avi
Type less by setting up aliases for Ubuntu terminal — October 5, 2020

Type less by setting up aliases for Ubuntu terminal

This method allows you to set a shortcut in the terminal for something you normally type out. For example, a long cd directory path that you normally need to, or a script you need to run that also requires input of a long directory.

To do this, edit ~/.bashrc, and add shortcuts on the bottom. The syntax is:

alias the_thing_you_type='the thing that gets done'

For example,

alias projects='cd /media/smirnovm/ssd2/Users/smirnovm'
Docker-Compose with Django setup —

Docker-Compose with Django setup

  1. in project root create a Dockerfile
  2. Make sure docker-compose is installed
  3. Add a docker-compose.yml file to the django project root
version: '3.7'

services:
  web:
    build: ./app
    command: python manage.py runserver 0.0.0.0:8000
    volumes:
      - ./app/:/usr/src/app/
    ports:
      - 8000:8000
    env_file:
      - ./.env.dev

4. Build it

docker-compose build

5. Run it. -d for detached, but better to see the console output for troubleshooting.

docker-compose up -d

6. View console output (in real time w/ -f tag) if you chose to run in detached mode

docker-compose logs -f
Create a virtual environment to run in Jupyter Notebook —

Create a virtual environment to run in Jupyter Notebook

To create a virtual environment in terminal:

python -m venv env

Then activate it

source env/bin/activate

Then install jupyter notebook integration. Replace “projectname” with whatever you want your environment to be called when selecting it in jupyter notebook

pip install ipykernel
ipython kernel install --user --name=projectname

Jupyter/GPU Dockerfile Template —

Jupyter/GPU Dockerfile Template

A Dockerfile template which works with nvidia-docker. Dockerfile is created in same directory as requirements.txt

Create Dockerfile in directory with requirements.txt 

FROM tensorflow/tensorflow:2.3.0-gpu-jupyter
RUN mkdir /tf/project
WORKDIR /tf/project
ADD requirements.txt .
RUN pip install -r requirements.txt

In terminal, run

docker build -t projectgroup/project . 

To run:

docker run -p 8888:8888 --gpus all -v ${PWD}:/tf/project projectgroup/project
PyCharm Python and Flask Live Templates Cheet Sheet! — February 7, 2020