Julia language is not ready (2018)

Update: I have picked up Julia again in 2020 and I am enjoying it.

Jupyter Error 49

$ jupyter notebook

Traceback (most recent call last):

File "/anaconda3/bin/jupyter-notebook", line 11, in


sys.exit(main())

File "/anaconda3/lib/python3.7/site-packages/jupyter_core/application.py", line 266, in launch_instance

return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)

File "/anaconda3/lib/python3.7/site-packages/traitlets/config/application.py", line 657, in launch_instance

app.initialize(argv)

File "", line 2, in initialize

File "/anaconda3/lib/python3.7/site-packages/traitlets/config/application.py", line 87, in catch_config_error

return method(app, *args, **kwargs)

File "/anaconda3/lib/python3.7/site-packages/notebook/notebookapp.py", line 1602, in initialize

self.init_webapp()

File "/anaconda3/lib/python3.7/site-packages/notebook/notebookapp.py", line 1381, in init_webapp

self.http_server.listen(port, self.ip)

File "/anaconda3/lib/python3.7/site-packages/tornado/tcpserver.py", line 143, in listen

sockets = bind_sockets(port, address=address)

File "/anaconda3/lib/python3.7/site-packages/tornado/netutil.py", line 168, in bind_sockets

sock.bind(sockaddr)

OSError: [Errno 49] Can't assign requested address

Short Answer

$ jupyter notebook --ip=127.0.0.1

PORT 8888

You can also explore what you are already running on PORT 8888

$ lsof | grep 8888Skype      407  uki  txt       REG                1,5   

$ lsof -nP | grep 8888

Julia

Download Julia

https://julialang.org/downloads/

For me, MacOS Package (.dmg)

               _

   _       _ _(_)_     |  Documentation: https://docs.julialang.org  (_)     | (_) (_)    |   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.  | | | | | | |/ _` |  |  | | |_| | | | (_| |  |  Version 1.0.1 (2018-09-29) _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release|__/                   |
julia> println("Hello World,", "Julia!")Hello World,Julia!

Clone Julia:

git clone https://github.com/JuliaLang/julia.git

Visit JuliaBox.com (iJulia Jupyter notebooks):

JuliaBox.com

Install Juno (Atom IDE)

http://docs.junolab.org/latest/man/installation.html

iJulia in Jupyter Notebook

julia> using Pkg
julia> Pkg.add("IJulia")

Jupyter NBInclude.jl

https://github.com/stevengj/NBInclude.jl

$ git clone https://github.com/stevengj/NBInclude.jl.git julia_NBInclude

Windows 10 UBuntu bash

My company uses mostly Windows and Ubuntu servers, I requested a MacBook, obviously.

However, there are certain internally-written programs that will run on the Windows only, so I run Parallels hypervisor with Windows 10 and Ubuntu.

On Windows 10, I have a hard time with the Command Prompt, last time I used DOS was in c. 1999, so I opted to try installing Bash.

$ ping Google.com

PING Google.com (74.125.138.102) 56(84) bytes of data.
64 bytes from yi-in-f102.1e100.net (74.125.138.102): icmp_seq=1 ttl=128 time=30.4 ms

$ ssh dummy@74.125.138.102

ssh: connect to host 74.125.138.102 port 22: Connection refused

Checking for Python

$ python --version

Command 'python' not found, but can be installed with:

sudo apt install python3
sudo apt install python
sudo apt install python-minimal

You also have python3 installed, you can run 'python3' instead.

$ python3 --version

Python 3.6.5

$ python3

Python 3.6.5 (default, Apr  1 2018, 05:46:30)

[GCC 7.3.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> print("Hello World")

Hello World

>>> exit()

$

Attempting to get Anaconda

wget https://3230d63b5fc54e62148e-c95ac804525aac4b6dba79b00b39d1d3.ssl.cf1.rackcdn.com/Anaconda-2.3.0-Linux-x86_64.sh
$ chmod +x Anaconda-2.3.0-Linux-x86_64.sh
$ . Anaconda-2.3.0-Linux-x86_64.sh

-bash: .: Anaconda-2.3.0-Linux-x86_64.sh: cannot execute binary file

$ java

Command 'java' not found, but can be installed with:

$ sudo apt install default-jre
$ sudo apt install openjdk-11-jre-headless
$ sudo apt install openjdk-8-jre-headless
$ sudo apt install openjdk-11-jre-headless

$ java --version

openjdk 10.0.1 2018-04-17
OpenJDK Runtime Environment (build 10.0.1+10-Ubuntu-3ubuntu1)
OpenJDK 64-Bit Server VM (build 10.0.1+10-Ubuntu-3ubuntu1, mixed mode)

Amazon remembers 30 of my past mobile devices!

It is funny, Amazon remembers 30 of my past mobile devices!

I had no idea until I have clicked for a new book to be delivered to my Kindle app.

Can you say I have been into mobile development since 2009?

graphlab.canvas.set_target('ipynb') error

I am getting an error in the following line:

graphlab.canvas.set_target('ipynb') # alternative 'browser', port=8889

/Volumes/DATA/anaconda3/envs/py27/lib/python2.7/site-packages/graphlab/canvas/server.pyc

108 self.__server = tornado.httpserver.HTTPServer(self.__application, io_loop=self.__loop)

TypeError: initialize() got an unexpected keyword argument 'io_loop'


Fix Attempt 1:
 

$ conda update tornado

environment location: /Volumes/DATA/anaconda3/envs/py27

tornado: 5.1-py27h1de35cc_0 --> 5.1.1-py27h1de35cc_0
Fix is not successful



Fix Attempt 2:

Reinstall Anaconda3

Conda environment as a Jupyter Notebook kernel

Conda Environment as a Jupyter Notebook kernel

Update conda

$ conda activate base
$ conda update -n base -c defaults conda
$ conda update --all -y

Add conda tensorflow environment

$ conda create -n py_36_tf python=3.6 tensorflow -y

List currently installed Jupyther kernels 

$ ls -alt ~/Library/Jupyter/kernels/

total 0
drwx------ 7 uki staff 224 Nov 22 11:50 ..
drwxr-xr-x 9 uki staff 288 Nov 22 11:43 .
drwxr-xr-x 5 uki staff 160 Nov 4 06:35 julia-1.4
drwxr-xr-x 5 uki staff 160 Apr 1 2020 python361064bitpy36condaa60168e76a7b4349b469299762ee4c30
drwxr-xr-x 5 uki staff 160 Apr 1 2020 python38264bitpytorchcondaaf5a833263b448b8b2738bb5a7355c8a
drwxr-xr-x 5 uki staff 160 Apr 1 2020 python361064bitturiconda565ecc262d0845fbb235ae21ac24296f

List conda environments you already created

$ conda info --envs

 

# conda environments:
#

/Users/uki/.julia/conda/3
base * /opt/anaconda3
py27 /opt/anaconda3/envs/py27
py36 /opt/anaconda3/envs/py36
py_36_tf /opt/anaconda3/envs/py_36_tf
pytorch /opt/anaconda3/envs/pytorch
tensorflow /opt/anaconda3/envs/tensorflow
turi /opt/anaconda3/envs/turi

Install new Jupyther kernel

$ python -m ipykernel install --user --name py_36_tf --display-name "Python 3.6 (tensorflow)"

Installed kernelspec py_36_tf in /Users/uki/Library/Jupyter/kernels/py_36_tf

Refresh a page with Jupyther Lab notebook and change to the new kernel.

Delete kernels that you want to replace

$ rm -r ~/Library/Jupyter/kernels/my_old_kernel_name

 

Microsoft Excel spreadsheet creates 1,048,576 rows!

I am sorry Microsoft, I really thought you have improved, but sometimes you just amaze me...

I bought my own Office license, so I am a paying customer.

The Excel creates 1,048,576 rows and yes, this brings the spreadsheet to a holt.

I am not making this up, there is an official post on it:

https://support.office.com/en-us/article/Locate-and-reset-the-last-cell-on-a-worksheet-C9E468A8-0FC3-4F69-8038-B3C1D86E99E9

The question is why in the world would you do it to a new spreadsheet with a few rows of actual data?

Force-killing frozen Excel

Re-opening the Excel reveals the same condition.

Select and Delete All Empty Rows

You can select and delete all rows below your last row with data.
- Select the first EMPTY row.
Depending on your keyboard you can use the following key combination:
- SHIFT + CONTROL + Arrow DOWN
or
- Control + END
- Right click and select DELETE

IT WORKS, then it recreated 1,048,576 rows!
The developer who created this functionality should be forced to do some very unpleasant things for each row created.

Hiding millions of rows and columns

Some people suggest to select and hide the offending rows and columns (to the right).
I have used it, but I think it is stupid, the rows are still there.

Select and Clear All Empty Rows

Since the cells were auto-generated, I am not sure why the article is suggesting this, but I tried this too.

- Select all empty cells (same as above)

- Right click and "Clear Contents"

- Save file

- Close

- Open file

No difference

Try #2

- Select all empty cells (same as above)

- Home > Clear > Clear All

- Save file

- Close

- Open file

No difference

The file on the disk is 29.5 MB

The Best Solution

- Select ALL POPULATED cells

- Open Apple Numbers

- paste cells

- Export file to Excel when you have to send it to coworkers

The file on the disk is 8 kB

Please share and donate!

Jupyter TOC

In order to create a Table of Contents in Jupyter Notebook, you need anchor tags above the headings:

replace_record
">

## Table of Contents

1. [Reading the CSV file](#Reading-CSV-file)
1. [The article's HTML template](#template)
1. [Main part - replace for every record](#replace_record)

Chrome Refresh

Caching of old content in the browser can drive you nuts when you develop HTML pages.

On Mac Google Chrome you refresh bypassing the cache by:

Holding both the ⌘ Cmd and ⇧ Shift keys and pressing the R key.

PCA9685

pip install Adafruit-PCA9685

References:

https://learn.adafruit.com/adafruit-16-channel-servo-driver-with-raspberry-pi/overview

L298N H-bridge Motor Controller

Connecting to Raspberry Pi Zero

Ubiquity Robotics Ubuntu Pi image for Pi with ROS

Ubiquity Robotics Lubuntu Pi image for Pi


On Ubuntu OS:
sudo apt install gnome-disk-utility

On Mac OS:
https://etcher.io/

Raspberry Pi 3 Model B
User :Ubuntu
Password: ubuntu (please change)




Pi HotSpot:

SSID is ubiquityrobotXXXX
where XXXX is part of the MAC address
Password: robotseverywhere

$ ssh ubuntu@192.168.50.16

The authenticity of host '192.168.50.16 (192.168.50.16)' can't be established.

ECDSA key fingerprint is SHA256:xq7SSuVwuVdifKaZGNqgt09LiDiZ0cnayd1tp4kWzD4.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '192.168.50.16' (ECDSA) to the list of known hosts.

ubuntu@192.168.50.16's password:

Welcome to Ubuntu 16.04.4 LTS (GNU/Linux 4.9.80-v7+ armv7l)

* Documentation: https://help.ubuntu.com

* Management: https://landscape.canonical.com

* Support: https://ubuntu.com/advantage

464 packages can be updated.

140 updates are security updates.
...


ubuntu@ubiquityrobot:~$ sudo apt-get update
[sudo] password for ubuntu:


ubuntu@ubiquityrobot:~$ sudo apt-get upgrade

E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?

Disable the Ubiquity Robotics scripts (unless you run their robots):
$ sudo systemctl disable magni-base
Removed symlink /etc/systemd/system/multi-user.target.wants/magni-base.service.

References:

https://ubuntu-pi-flavour-maker.org/

Ubiquity Robotics Lubuntu Pi image for Pi:

https://downloads.ubiquityrobotics.com/pi.html

Python template

To replace tokens in the HTML template I use method str.format()

line3 = '      {ARTICLE_TITLE}'
line3 = line3.format(ARTICLE_URL="http...", ARTICLE_TITLE="My Note")
print( line3 )

My Note'

GitHub gitignore

Sometimes the .gitignore file does not work with GitHub, to fix it I execute the following:



$ git rm -r --cached . 

$ git commit -a -m "fixed .gitignore"

Remove commit

Sometimes when you COMMIT to GitHub a file larger than 100MB, the PUSH is rejected.


Your branch is ahead of 'origin/master' by 5 commits.
You have to remove the last (or more) COMMIT, the following command does not change your code:

git reset --soft HEAD~1

Then fix the problem, usually by marking the large files in .gitignore

$ git commit -a -m "added gitignore with _DATA/* directory"


$ git status

Untracked files:
(use "git add ..." to include in what will be committed)

src/_DATA/amazon_baby.csv
src/_DATA/amazon_baby.gl/
src/_DATA/home_data.gl/
src/_DATA/image_test_data/
src/_DATA/image_train_data/
src/_DATA/people-example.csv
src/_DATA/people_wiki.csv
src/_DATA/people_wiki.gl/
src/_DATA/song_data.csv
src/_DATA/song_data.gl/


Then push again and pray:

$ git push

 $ git status

On branch master

Your branch is up to date with 'origin/master'

GitHub SSH

Go to:

https://github.com/settings/keys

Read:

https://gist.github.com/developius/c81f021eb5c5916013dc


$ ssh -T git@github.com


The authenticity of host 'github.com (192.30.253.112)' can't be established.

RSA key fingerprint is SHA256:nTh....

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'github.com,1xx.xx.xx.xx' (RSA) to the list of known hosts.

Enter passphrase for key '/Users/uki/.ssh/id_rsa':

Hi UkiDLucas! You've successfully authenticated, but GitHub does not provide shell access.

Find files that are too big for GitHub

Command explanation

• Find
• in current directory
• files bigger than 50 MB, (100Mb is too big for GitHub)
• edited within 365 days
• the type will be a file
• execute list files
• list format
• human readable
• sort by Size (not working)

$ find . -size +50M -mtime +365 -type f -exec ls -lhS {} \;

-rw-r--r--@ 1 uki  admin    49M Nov  6  2016 ./Coursera/UW/ML/Week3/amazon_baby.gl/m_bfaa91c17752f745.0000

-rw-r--r--@ 1 uki  admin    69M Nov  6  2016 ./Coursera/UW/ML/Week4/people_wiki.gl/m_4549381c276b46c6.0000

-rw-r--r--@ 1 uki  admin    60M Nov  6  2016 ./Coursera/UW/ML/Week5/song_data.gl/m_cccc16853452d1ed.0000

-rw-r--r--@ 1 uki  admin    63M Nov  6  2016 ./Coursera/UW/ML/Week6/image_test_data/m_e16f5ffd2c088370.0000

-rw-r--r--@ 1 uki  admin    31M Nov  6  2016 ./Coursera/UW/ML/Week6/image_train_data/m_504edbda459b24ff.0000

ROS Mac

http://wiki.ros.org/kinetic/Installation/OSX/Homebrew/Source

ROS Droid

Creating Android ROS nodes to:
- add control UI (HMI)
- utilize existing phone sensors:
-- gyroscope
-- GPS
-- compass
-- camera
- do some data processing (flagship phones are very powerful)
- use TensorFlow AI

References:
https://github.com/rosjava/android_core

http://wiki.ros.org/rosjava/Tutorials/Create%20a%20ROS%20Android%20Node

HiKey 960 ver.B Single Board Computer (SBC) 4GB

Specifications:

• Kirin 960 SOC, Big.Little architecture
•     4 Cortex A73 CPU
•     4 Cortex A53 CPU 
• GPU: Mali G71 MP8 (high image processing)
• 4GB LPDDR4 DRAM
• 32GB UFS flash storage
• Power: 12V@2A power supply (8v to 18V)
• DC jack (EIAJ-3 compliant)
• 4.75mm outer diameter
• 1.7mm center pin (positive)
• 2 x USB 3.0 Type-A
• 1 x USB Type-C OTG (if anything is plugged-in the other USB stop working)
• Audio/Video OUT: HDMI 1.4 ( Type A-full) monitor connector
• Micro SD card
• Wi-Fi TIWL1837MOD 2.4GHz, 5GHz (dual-band, dual antenna)
• Bluetooth
• PCIe Gen 2 M.2 M key connector
• External boot switch
• Boot mode switch
• Auto power switch
• Power reset button
• 40-pin connector (low speed): 
• I2S, 
• 2 x UART, 
• 2 x I2C
• SPI
• 12 x GPIO
• DC power
• 60-pin HS (high-speed) connector
• 4L-MIPI DSI
• I2C x2
• SPI (48M)
• USB 2.0
• 2L+4L MIPI CSI
• OS: Android / Linux

Getting Started page:
https://www.96boards.org/documentation/consumer/hikey/hikey960/getting-started/


Factory Image (latest hikey960 linaro):
~ 600MB zip
local storage: /Volumes/DATA/linaro/HiKey/image/hikey960-linaro-2018.10.04-factory-ccfcf697.zip
https://snapshots.linaro.org/96boards//hikey960/linaro/aosp-master/latest/

Good video on building AOSP for HiKey 960:

https://youtu.be/RD_NZRKiZBU

Android Code Tree:
https://gitlab.com/HiKey960-Car/android_device_linaro_hikey/tree/master_091118_car96


When USB C is connected, neither of USB A will work:
https://bugs.96boards.org/show_bug.cgi?id=500#c1

DBPedia

Exploring..


https://wiki.dbpedia.org/

Jupyter notebook TOC

https://github.com/shoval/jupyter-navbar

Python Dictionary (KEY, VALUE pairs) operations

# define empty dictionary
special_words_count = {}


# populate dictionary with sums
special_words_count["awesome"] = products["awesome"].sum()
special_words_count["great"] = products["great"].sum()
special_words_count["fantastic"] = products["fantastic"].sum()
special_words_count["amazing"] = products["amazing"].sum()
special_words_count["love"] = products["love"].sum()
special_words_count["horrible"] = products["horrible"].sum()
special_words_count["bad"] = products["bad"].sum()
special_words_count["terrible"] = products["terrible"].sum()
special_words_count["awful"] = products["awful"].sum()
special_words_count["wow"] = products["wow"].sum()
special_words_count["hate"] = products["hate"].sum()



# print values in descending order
# key=lambda x: x[1] # gets the VALUE (i.e. the second [0,1] element) of the PAIR x.
# reverse=True - set descending order
for key, value in sorted(special_words_count.items(), key=lambda x: x[1], reverse=True):
    print value, key

42420 great
40277 love
3197 bad
2002 awesome
1305 amazing
1057 hate
873 fantastic
673 terrible
659 horrible
345 awful
131 wow

Another example of sorting the Dictionary

print name_value_dictionary

{'fantastic': 0.8913030903042991, 'love': 1.3998983430174596, 'bad': -0.9858273699287683, 'awesome': 1.0580088887848043, 'great': 0.8839378948979554, 'terrible': -2.090499984872614, 'amazing': 0.8928024225084649, '(intercept)': 1.367283152293053, 'horrible': -1.9965180055878113, 'awful': -1.7646995563132248, 'hate': -1.4091640627563906, 'wow': -0.0541450123332599}



def input_method(my_tuple):
    #print "*", my_tuple[1] # sanity check
    return my_tuple[1]

tuples = name_value_dictionary.items() # key: value

for key, value in sorted(tuples, key=input_method, reverse=True):
    print value, key






1.39989834302 love
1.36728315229 (intercept)
1.05800888878 awesome
0.892802422508 amazing
0.891303090304 fantastic
0.883937894898 great
-0.0541450123333 wow
-0.985827369929 bad
-1.40916406276 hate
-1.76469955631 awful
-1.99651800559 horrible
-2.09049998487 terrible