27 February 2018

Learning about set theory

I recently had a problem thinking up a solution for a tricky database query. I'll use the common movie database paradigm to demonstrate it.

A registered user is able to store their preferred genres, so when they search for a movie by preferred genre, they can filter out the movies that don't appear in their preferred genres. As you know, you can have more than one preferred genre, and movies can be multiple genres, for example, romantic comedy. The requirement here, for a user that likes action, comedy, horror, and sci-fi, is not to see romantic comedies in their results because although they like comedies, they haven't specified romantic films on their list. If they wanted to see romantic comedies on their list they would have to also add the genre romantic to the list.

Ok, so the requirements are done with. How do we implement it. Well I got quite stuck to be honest. I started searching, but found it difficult to find a pre-written query for just what I needed. After Googleing "how to think in SQL", I then went on to explore set theory, which I remember doing something with in school, but didn't pair with databases, as people never really even had computers in their home during this time. So I didn't even know what a database was.

Doing some searching, I found this great YouTube tutorial on set theory. During it, I realised I could perhaps get what I needed by subtracting one set from another. Let's look at the tables first though.

[movies]---<[movies_genres]>---[genres]---<[users_genres]>---[users]

In our normalised table structure, we are comparing the foreign key genres_id found in pivot tables movies_genres and users_genres.

Looking at figure 1, if we imagine one movie and one user, you can see we cannot subtract the set movies_genres.genres_id from users_genres.genres_id because the genres_id 3 is left over in movies_genres.genres_id. So movies_genres.genres_id is not a subset of users_genres.genres_id.

Example: Figure 1.

movies_genres.genres_id {1,2,3} 
users_genres.genres_id  {1,2}

In figure 2, we can subtract the set movies_genres.genres_id from users_genres.genres_id because all numbers appear in movies_genres.genres_id that appear in users_genres.genres_id. The 3 you can see in users_genres.genres_id is fine as we don't need to exactly match the query. Here movies_genres.genres_id is a subset of users_genres.genres_id because all numbers appear in the second set.

Example: Figure 2.

movies_genres.genres_id {1,2} 
users_genres.genres_id  {1,2,3}

So how do you test if one set is a subset of another set in MySQL? I've used the in() operator. I'll quote W3Schools for the explanation.

"The IN operator allows you to specify multiple values in a WHERE clause. The IN operator is a shorthand for multiple OR conditions."

To use the in() operator for subtraction, I negate it with not in(). This tests whether there are values left over when comparing genres_id between movies_genres and users_genres (see Figure 1.). This is essentially the subset check. When we can test to see which queries have failed the subset check, we can negate the results to select the movies we want.

So with that, this is how I do it in MySQL.

select * from movies 
where movies.id not in (
    select movies_id from movies_genres mg
    where mg.genres_id not in (
        select genres_id from users_genres ug 
            where ug.users_id = 2
    )
)

Conclusion

I hope this is correct. I am quite inexperienced with SQL and never really learned it in an academic environment. I hope to solve more difficult MySQL problems with set theory in the future.

28 January 2018

Mining Magi Coin on the Raspberry Pi Model B Revision 1

This is how I did it. Disclaimer, I'm very new to this. Some info might be incorrect.

Setup the Pi

Download and install Raspbian Lite (Raspian without the GUI).

Add a file called ssh (no extension) to the root of the SD card. This will enable SSH at boot.

Connect the rp to a client PC over ssh.

  • Boot the rp, but don't connect anything yet
  • In the Network settings of your client PC, select Edit connections (click the network icon to access these settings)
  • Select Add
  • Select Ethernet
  • Change the connection name to something like Pi.
  • In the IPv4 Setting, set the Method to Shared to other computers
  • In the IPv6 Setting, set the Method to Ignore
  • Click Save
  • Run sudo apt-get install arp-scan on the client PC.
  • Uncheck Enable Wifi in network settings, or however you connect to the internet.
  • Plug the ethernet cable into the rp (directly connecting it to the PC client)
  • Select Pi from network tab. A confirmation message Connected to Pi should appear
  • Run sudo arp-scan -l to get the network address of the rp
  • In case you get an error (ioctl: No such device) you might have to choose your network device manually, get a list of all devices installed with this command ifconfig. Depending on this info you maybe have to specify something like eth2 Do this by altering the above command with the --interface option like this sudo arp-scan --interface eth2 -l

Now follow the security procedure here https://www.raspberrypi.org/documentation/configuration/security.md

Make other rp configurations sudo raspi-config. Select boot to console. Reduce the GPU to 16MB.

Run locale to see output from the locale list. If the first two items are missing their values, in ~/.bashrc (at the end of the script) add

LANG=en_GB.UTF-8
LANGUAGE=en_GB 

Or whatever your preferred language is. You can now connect the rp to the internet, although I ssh into it over the local network and don't open any ssh ports for external access. Run sudo apt-get update && sudo apt-get upgrade to make sure everything is updated. Reboot sudo shutdown -r now. You may also want to statically set the address of the rp on the router.

Install the cpu miner.

Modified from this guide (this guy is using a RP2): https://www.novaspirit.com/2017/10/19/crypto-mining-sbc/

Satisfy any dependencies.

sudo apt-get install automake autoconf pkg-config libcurl4-openssl-dev libjansson-dev libssl-dev libgmp-dev make g++ git libgmp-dev

Clone the git source.

git clone https://github.com/novaspirit/wolf-m7m-cpuminer

Compile the miner.

cd wolf-m7m-cpuminer 
./autogen.sh

Run the rp model B rev 1 compiler specific flags.

CFLAG="-O2 mfpu=vfp" ./configure

Modify 2 make files with rp model B rev 1 compiler specific flags

nano Makefile

Do Ctrl + w and search for march. Replace -march=native with -mcpu=arm1176jzf-s for manual cpu detection.

nano m7/Makefile

Do Ctrl + w and search for march. Replace -march=native with -mcpu=arm1176jzf-s for manual cpu detection.

Run make to build the miner.

The miner is now installed. Once you have joined a mining pool, you can run something like the the following

./minerd -a m7mhash -o urlofpool:5678 -u pool_username.worker_name -p worker_password

But I prefer to create a file called miner.sh in the home directory.

cd ~/
touch miner.sh
chmod +x miner.sh
nano miner.sh

Add (replace the tcp address and credentials with your pool's info)

#!/bin/bash 
nohup ./wolf-m7m-cpuminer/minerd -a m7mhash -o stratum+tcp://xmg.suprnova.cc:7128 -u pool_username.worker_name -p worker_password &

Using nohup <command> & allows you to log out of the ssh session and keep the miner running. Save and close the file. In .bashrc, add the alias alias magi='miner.sh', alias magi-mon='tail -f nohup.out' and alias magi-kill='pkill minerd'

You can now run the miner with magi, and monitor the miner with magi-mon, or magi-kill to end the miner process during any session. To quit magi-mon (tail output), call Ctrl + c

Get the wallet

To get a wallet receiving address (to receive mined or donated coin), you will need a wallet. For magi coin, a wallet is an application you need to install. Get it here http://www.m-core.org/download/

After installing the wallet application, you will need to get something called the blockchain to have the wallet application create your receiving address. This is a very big file and the app will try and download it by default when you launch it. However, there is a much quicker way to get this.

  • Backup wallet.dat to a safe place.
  • Download fairly recent block-chain data from here: http://m-core.org/bin/block-chain
  • Delete all of the contents in the ~/.magi folder, except for wallet.dat and magi.conf
  • Unzip the block-chain file and copy the folders under "m-block-chain" into the .magi (unix-like system) or Magi (OS X or Windows) folder
  • Launch the new wallet.

This will quickly get you a complete (but old) block-chain so the app can create you a receiving address. The app will now attempt to catch up to the latest version of the block-chain, which will help you get your latest mined coins (those coins exist on the block-chain). You can now find the receiving address in the Received tab of the wallet application.

Join a pool

The recommended way to mine magi coin is with a mining pool. Here is a list of pools https://poolinfo.systms.org I heard it's best to join a pool with a high hashrate, which is what I did. Register with one of these pools, and check out the settings in My Account > Edit Account.

Next you will need to create a worker (My Account > My Workers). Fill in the required fields and make a note of the details for the next step (the pool address). Back when we created the miner, there was some parameters you had to pass

./minerd -a m7mhash -o urlofpool:5678 -u pool_username.worker_name -p worker_password

Go to Help > Getting Started and check "Configure your miner" on that page of the pool site, and edit your miner parameters to reflect how you execute your miner on the rp.

Mine looks like

nohup ./wolf-m7m-cpuminer/minerd -a m7mhash -o stratum+tcp://xmg.suprnova.cc:7128 -u pool_username.worker_name -p worker_password &

Finishing up

Using a Raspberry Pi Model B Revision 1, I managed to get about 0.34 khash/s. For comparison, I hear a rp 2 get's about 5 khash/s and I think it even uses less power! I haven't done the math yet, but I don't think this is profitable or fast enough to be worthwhile. It has been a fun learning experience though.

29 October 2017

Get a Dart-lang web dev environment installed for Visual Studio Code

This is for Windows, but it's pretty easy to figure out for Linux too.

Open Visual Studio Code and use the addon installer to install the Dart Code plugin from Danny Tuppeny

From the Dart site, download and install the dart-sdk.

Put the location of the sdk on your $PATH variable

C:\Program Files\Dart\dart-sdk\bin

Use Dart's pub tool (Dart's package manager) to install stagehand (the Dart project generator). In the terminal run

pub global activate stagehand

Then add the following to you $PATH (replacing your_username)

C:\Users\your_username\AppData\Roaming\Pub\Cache\bin 

You will be using Dartium you view your web apps. Dartium is bundled with the Windows install of the SDK. You can find it in

C:\Program Files\Dart\chromium\chrome.exe 

Create a shortcut for convinience. Note: Dartium will no longer be needed from Dart 2.0.

To run a test app, create a test folder and open it in VS Code. Open the VS Code terminal and run

stagehand web-simple

Your test folder should be populated with the broilerplate dev files.

Provision the required packages by running

pub get

To launch your test app run

pub serve

Note the response (mine was http://localhost:8080/) and open that in the Dartium browser. You should see Your Dart app is running. You can still test apps in any other browser by opening the same address in that browser, but it may take a several seconds to build (check the terminal for feedback).

30 August 2017

Basic commands to quickly set file/folder permissions and ownership

Assuming you have navigated to the directry you want these changes to take effect ie. /var/www/your_vh_domain

Note: Remember to set individual restricted permissions to sensitive files and folders after this bulk action has taken place. For example, with WordPress you will want to set wp-config.php file to 600

Change folder permissions recursively

find public_html -type d -exec chmod 755 {} \;

Change file permissions recursively

find public_html -type f -exec chmod 644 {} \;

Change file ownership recursively - Set your own user and group as desired

chown -R leke:www-data public_html

20 August 2017

Sites that offer free stock images

Here is a collection sites that offer free stock images you can use for your projects. Please check the current licensing status for each site as it might have changed as of writing this.

Unsplash

Unsplash - Beautiful, free photos. Gifted by the world’s most generous community of photographers

The Stocks

http://thestocks.im/ - the best royalty free stock photos in one place

Splitshire

https://www.splitshire.com/about/ - Helping you to be awesome out there with great content.

Pic Jumbo

https://picjumbo.com/ - picjumbo is free stock photo site created by designer & photographer Viktor Hanacek in 2013.

Gratisography

http://gratisography.com/ - Free high-resolution pictures you can use on your personal and commercial projects. New awesome pictures added weekly! All pictures were photographed by Ryan McGuire and free of copyright restrictions.

Startup Stock Photos

http://startupstockphotos.com/ - Free photos for startups, bloggers, publishers, websites, designers, developers, creators, & everyone else.

Negative Space

https://negativespace.co/ - Beautiful, Free High-Resolution Photos with No Restrictions For personal or commercial projects, all of our CC0 licensed images are completely free to use!

Life of Pix

http://www.lifeofpix.com/ - Free, high resolution photography.

Endless Icons

http://www.endlessicons.com/- Endless Icon is a project bulid by Min Kim who runs Email-Gallery, App Monstr, and Endless Photos. I created it to share some of the icons and creative stuff with the design community. Cheers!

26 July 2017

Get a Windows licence key from a machine after it's been uninstalled

It turns out you can get the Windows licence key from a machine, even after it's been uninstalled, using Linux. Run this in the console

sudo hexdump -s 56 -e '"MSDM key: " /29 "%s\n"' /sys/firmware/acpi/tables/MSDM

If the key is in your machine's firmware, you should see it.

15 July 2017

I've been using the wrong date delimiters in PHP for JavaScript

So I've been making dates in PHP like this

$dt = new DateTime("2017-07-15");
$date = $dt->format("Y-m-d");

and passing that to JS like this

echo "foo = new Date(\"$date\");";
// returns: Sat Jul 15 2017 00:00:00 GMT+0300 (FLE Summer Time)

and that was working fine in Chrome and Firefox. However, if you try and call foo in Safari, you'll get an invalid date error. This is because Safari doesn't currently support the dash delimiter format. More info.

To avoid this in the future, I need to start remembering to format my dates in php with / instead of - when I'm going to be using it in the web-page. So like this

$dt = new DateTime("2017/07/15");
$date = $dt->format("Y/m/d");

The only thing I can see still needing dashes, would be in URLs. If you need to switch to dashes while in JS, you can do something like .replace(/\//g, "-")

I've been neglecting testing in Safari which is pretty bad. Luckily, you can still download an old version of Safari for Windows, so testing is easy enough.

24 June 2017

CodeIgniter 3 Sessions

CI sessions are an alternative to php sessions. It is advised to autoload the sessions so they are always available. CI sessions simplify using native php sessions, ie the $_SESSION superglobal. For example, there is no need to include the session_start() function (to start a new or resume an existing session). Session data is simply an array, associated with a particular session ID (cookie).

As well as read, set and unset values, CI also provides 2 special types of session data, flashdata and tempdata.

To read a session variable use either $_SESSION['item_key_name'] (or the magic getter) $this->session->item_key_name

To retrieve all of the existing userdata, you can simply omit the item key $_SESSION for example print_r($_SESSION) (the magic getter only works for properties).

You can assign (set) data to the $_SESSION array, as with any other variable. Or as a property of $this->session. For example $_SESSION['something'] = 'hello' or $this->session->something = 'hello'

Check if a session variable exists: isset($_SESSION['some_name'])

To remove session variables, use for example, unset($_SESSION['some_name']) or multiple values: unset($_SESSION['some_name'], $_SESSION['another_name'])

Session Flashdata

Flashdata is data that will only be available for the next request, and is then automatically cleared. This can be very useful, especially for one-time informational, error or status messages. To create a session variable as flashdata:

$_SESSION['item'] = 'value';
$this->session->mark_as_flash('item');

Or more concisely:

$this->session->set_flashdata('item', 'value');

Read flashdata variables the same way as regular session data $_SESSION['item']

However, if you want to be sure that you’re reading “flashdata” (and not any other kind), you can also use the flashdata() method:

$this->session->flashdata('item');

Or to get an array with all flashdata, simply omit the key parameter:

$this->session->flashdata();

If you find that you need to preserve a flashdata variable through an additional request, you can do so using the keep_flashdata() method.

$this->session->keep_flashdata('item');

Tempdata

Tempdata is session data with a specific expiration time. To mark an existing item as tempdata, simply pass its key and expiry time (in seconds!) to the mark_as_temp() method:

$_SESSION['item'] = 'value';
$this->session->mark_as_temp('item', 300); // erased after 300 seconds

Or concisely:

$this->session->set_tempdata('item', 'value', 300);

Read tempdata variables the same way as regular session data $_SESSION['item']

Destroying a Session

To clear the current session both of these will work in exactly the same way session_destroy() or $this->session->sess_destroy()

Conclusion

You can still use the native php session functions with the CI session library and for simplicity, I think this is best practice. You then only need to learn the extra CI session methods (ie. flashdata, tempdata). The added bonus to the CI's session library is that you don't have to add session_start() to every file that uses sessions.

30 May 2017

CodeIgniter's last_query()

I had an issue running codeigniter's $this->db->last_query().

It returns the last query that was run (the query string, not the result). But, remember to run the query, ie call the get() method before the last_query() method.

Example

// Some query you are having trouble building...
$this->db->select('a_column');
$this->db->from('your_table');
// note, you have to call get()
$query = $this->db->get();
echo $this->db->last_query(); // output the converted MySQL query 

This method is helpful when having trouble building queries with the CI query class.

25 April 2017

Running Vagrant on Windows

Recently Packt Publishing kindly gave away Learning PHP 7 in a short 24 hour promotion. I've been going through it and had a problem setting up Vagrant 1.9.3 on Windows. Specifically running vagrant up returned this error

C:/Program Files (x86)/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.9.3/lib/vagrant/util/is_port_open.rb:21:in `initialize': The requested address is not valid in its context. - connect(2) for "0.0.0.0" port 8080 (Errno::EADDRNOTAVAIL) 

Here is the reported error and fix Vagrant 1.9.3 up fails.

None of this is the book's falt of course. Technology changes fast and it's tough to keep up to date with it.

Next problem I had was the book assumes you have an ssh client installed on windows, which I didn't, so the command vagrant ssh returned another error. Although this time, the error simply instructed me to install an ssh client and even suggested some. I chose Git, but found there was one more step to do after installing it. You need to add C:\Program Files\Git\bin and C:\Program Files\Git\usr\bin to your Windows environment path. This latter path contains ssh.exe along with a bunch of other tools.

Now you can easily run vagrant up and vagrant ssh from your project folder path.

The chapter index in this book looks fantastic! There are lots of new things I need to learn there, and some other things I should master. Good luck.