A Mouse Follower

Table of Contents

Middle

let mouse_follower_sketch = function(p) {
    p.setup = function() {
        let parent_div_id = "mouse-follower";
        this.canvas = p.createCanvas($("#" + parent_div_id).outerWidth(true), 800);
        //this.canvas.parent(parent_div_id);
        p.walker = new Walker(p);
    }

    p.draw = function() {
        p.background(255);
        p.walker.walk();
        p.walker.display();
    }
};

function Walker(p) {
    this.position = p.createVector(p.width/2, p.height/2);
    this.velocity = p.createVector(0, 0)

    this.walk = function() {
        mouse = p.createVector(p.mouseX, p.mouseY);
        // calling sub on the vectors does an in-place update
        // using p5.Vector.sub creates a new vector
        // This is a static method so we use the module (p5) not the instance (p)
        acceleration = mouse.sub(this.position);

        // setMag always produces the same magnitude (but the orientation stays the same)
        acceleration.setMag(0.5);
        this.velocity = this.velocity.add(acceleration)
        this.position = this.position.add(this.velocity)
  }

  this.display = function() {
      p.stroke(0);
      p.noFill();
      p.background(255, 255, 255, 25);
      p.ellipse(this.position.x, this.position.y, 48, 48);
  }
}

sketch_container = new p5(mouse_follower_sketch, 'mouse-follower');

End

  1. Shiffman D. The nature of code: simulating natural systems with processing. Version 1.0, generated December 6, 2012. s.l.: Selbstverl.; 2012. 498 p.

Processing Test

Table of Contents

This is a p5 test based on their get started tutorial. It's also an indirect test of using the raw directive to write HTML to pass to nikola.

And now the sketch. If you run your mouse over the blank area you should see a bunch of circles following the cursor.

<div id="get_started">
    <script language="javascript" type="text/javascript" src='get_started.js'></script>
</div>
var diameter;

function setup() {
    canvas = createCanvas(0.8 * windowWidth, 200);
    canvas.parent("get_started")
    background(255);
    strokeWeight(3);
    stroke(0, 0, 255);
    fill(255);
}

function mousePressed() {
    /* set background to blue */
    background(0, 0, 255);
}

function mouseReleased() {
    /* set background to white */
    background(255);
}

function draw() {
    /* Draw circles that change diameter based on mouse speed */
    /* and color based on if mouse-pressed (or not pressed)   */
    if (mouseIsPressed) {
        fill(0, 0, 255);
        stroke(255);
    } else {
        fill(255);
        stroke(0, 0, 255);
    }
    diameter = pow(dist(pmouseX, pmouseY, mouseX, mouseY), 1.5);
    ellipse(mouseX, mouseY, diameter, diameter);
}

Notes

This was a little harder than I thought it would be. First, regarding the javascript file:

  • In order to get it to the final HTML you need to put in the listings folder at the root of the nikola folder and use the listing reStructuredText directive instead of include (this is a special nikola directive).

And then there's the html trick:

  • to get the sketch to stay in the post (instead of showing up on the bottom of the page), you have to use the div trick - create a div where you want the sketch to be and give it a unique ID (use the raw reStructuredText to put the HTML tags displayed above into the post), then set the parent in the sketch (canvas.parent("get_started") in this example).

A Random Accelerator

Table of Contents

Beginning

This is an extension of the random walker with acceleration added.

Middle

let random_accelerator_sketch = function(p) {
    p.setup = function() {
        let parent_div_id = "random-accelerator";
        this.canvas = p.createCanvas($("#" + parent_div_id).outerWidth(true), 800);
        this.canvas.parent(parent_div_id);
        p.walker = new Walker(p);
    }

    p.draw = function() {
        p.background(255);
        p.walker.walk();
        p.walker.display();
    }
};

function Walker(p) {
    this.position = p.createVector(p.width/2, p.height/2);
    this.velocity = p.createVector(0, 0)

    this.walk = function() {
        acceleration = p.createVector(p.random(-1, 1), p.random(-1, 1));
        acceleration = acceleration.mult(0.1)
        this.velocity = this.velocity.add(acceleration)
        this.position = this.position.add(this.velocity)
  }

  this.display = function() {
      p.stroke(0);
      p.noFill();
      p.background(255, 255, 255, 25);
      p.ellipse(this.position.x, this.position.y, 48, 48);
  }
}

sketch_container = new p5(random_accelerator_sketch, 'random-accelerator');

End

This was a very rudimentary walker, the main point of it was that at this point we have the basic kinematic elements to make something following the rules of classical physics (more or less).

  1. Shiffman D. The nature of code: simulating natural systems with processing. Version 1.0, generated December 6, 2012. s.l.: Selbstverl.; 2012. 498 p.

Random Walker Take Two

Table of Contents

Beginning

Another take on getting p5.js to behave using the random walker.

Middle

First, make a div to contain the javascript.

<script language="javascript" type="text/javascript" src='random-walker-vectorized.js'></script>          
    <div id="random-walker-vectorized">
</div>

Then the javascript.

let random_walker_sketch = function(p) {
    p.setup = function() {
        let parent_div_id = "random-walker-vectorized";
        this.canvas = p.createCanvas($("#" + parent_div_id).outerWidth(true), 300);
        this.canvas.parent(parent_div_id);
        p.walker = new Walker(p);
    }

    p.draw = function() {
        p.background(255);
        p.walker.walk();
        p.walker.display();
    }
};

function Walker(p) {
    this.position = p.createVector(p.width/2, p.height/2);

    this.walk = function() {
        velocity = p.createVector(p.random(-5, 5), p.random(-5, 5));
        this.position = this.position.add(this.velocity)
  }

  this.display = function() {
      p.stroke(0);
      p.noFill();
      p.background(255, 255, 255, 100);
      p.ellipse(this.position.x, this.position.y, 48, 48);
  }
}

sketch_container = new p5(random_walker_sketch, 'random-walker-vectorized');

End

The big thing that fixed the positioning of the canvas was that I was putting the line to include the javascript inside the div where it was supposed to render. Keeping the div for the canvas empty fixed it.

A Random Walk(er)

Beginning

This is another post to see if I understand how to get p5.js working in nikola. It's been a while since I tried and I just want to see if I remember how. This uses the random walk example from Daniel Schiffman's book the Nature of Code.

Middle

A Div to Locate the Sketch

The id of this div is set in the p5.js setup function as the parent of the sketch.

<script language="javascript" type="text/javascript" src="walker.js"></script>
<div id="random-walk-container">
</div>

Note: Originally this wasn't working, because I had the line to include the javascript inside the div to hold the canvas. Make sure that div is always empty.

The Javascript

let sketch = function(p) {
    p.setup = function() {
        let parent_div_id = "random-walk-container";
        this.canvas = p.createCanvas($("#" + parent_div_id).outerWidth(true), 300);
        this.canvas.parent();
        p.walker = new Walker(p);
    }

    p.draw = function() {
        p.background(255);
        p.walker.walk();
        p.walker.display();
    }
};

function Walker(p) {
  this.x = p.width/2;
  this.y = p.height/2;

  this.walk = function() {
    this.x = this.x + p.random(-1, 1) * 10;
    this.y = this.y + p.random(-1, 1) * 10;
  }

  this.display = function() {
    p.fill(0);
    p.ellipse(this.x, this.y, 48, 48);
  }
}

//let node = document.getElementById("random-walk")
//window.document.getElementsByTagName("body")[0].appendChild(node);
sketch_container = new p5(sketch, 'random-walk-container');

End

As always, this was way harder than it should have been.

Moving BeyondPod Files To the SDCard (Android 9)

Introduction

I don't follow the changes going on with Android closely enough to know exactly when all the changes were made to disable moving things to the SD Card, but I've been running out of storage recently, even though my SD Card has over 60 GB of free space so I looked into it and there are currently three things that seem to have changed that caused this problem with my Moto-X running Android Pie (9):

  • The option to use the SDCard as an extention of the Internal Storage has disappeared from the storage options.
  • The option to move any of my apps to the SDCard has disappeared from the Apps' settings
  • The last update seems to have broken all the connections between my apps and the SDCard so none of them are (were) using the external storage

There might be a way to get around the first two problems, but I don't really feel like chasing that right now. It turns out that fixing the last case for some of my apps works, but isn't as intuitive as I would like it to be. Here's what to do.

Give Your App Storage Permissions

Settings

In your Android settings menu pick "Apps & notifications".

apps_and_notifications.png

Apps & Notifications

Next pick Beyond Pod from the list of applications (in this case it was one of my recently opened applications, but it isn't always).

apps_list.png

App Info

In the BeyondPod settings make sure that Permissions has Storage listed, if not tap the Permissions to get to that setting.

beyond_pod_settings.png

App Permissions

In the "App permissions" make sure the switch next to "Storage" is turned on.

beyond_pod_storage.png

Figure Out The Path To Your SDCard

Using ADB

I couldn't any way to find out the path to the sdcard in the Android settings themselves. The easiest way (to me) is to set up android debug bridge and then list the contents of the storage folder.

hades@erebus ~/d/datasets [1]> adb shell
payton_sprout:/ $ ls /storage/
ls: /storage//193D-4160: Permission denied
56DC-7D9D emulated self 
payton_sprout:/ $ df -h storage/56DC-7D9D/
Filesystem              Size  Used Avail Use% Mounted on
/mnt/media_rw/56DC-7D9D  60G  3.5G   56G   7% /storage/56DC-7D9D

The Wrong Way

If you look at the permissions for the folder you can see that the folder itself has read-write permissions if you're root or part of the sdcard_sw group

payton_sprout:/ $ ls -l storage/56DC-7D9D/
ls: storage/56DC-7D9D//.android_secure: Permission denied
total 128
drwxrwx--x 3 root sdcard_rw 131072 2019-01-27 14:04 Android

and although there is that Permission denied for the .android_secure file, it let me create folders and files in there so I figured I would create a folder for downloads and point BeyondPod to it.

It turns out that this doesn't work. I was going to walk through the error but I've already set it up the right way and I don't want to undo it. The key to figuring out why it kept telling me my folder didn't exist or was read only was finding this beyondpod forums thread. It looks like when you give permission to BeyondPod to use the SDCard, Android creates a specific folder that BeyondPod can use and you have to point it there. The format is:

/storage/<sd card>/d/data/mobi.beyondpod/files/

So in my case the path is:

/storage/56DC-7D9D/Android/data/mobi.beyondpod/files/

Using "Files" Instead of ADB

Even though the Settings menus don't seem to show you the path to the sdcard you can use a file browser app if you don't want to use adb. Here's my sdcards name in the files app (it's not a file browser, but it's rather something that's supposed to help you clean up your storage but it works for this case).

files_browser.png

Last Step: Point Beyond Pod To the SDCard

Settings

Open Beyond Pod and scroll to the bottom feeds list and tap on the Settings option.

beyond_pod_settings_menu.png

Advanced Settings

Now click on the hamburger menu icon on the top right to open it up and tap on Advanced Settings.

beyond_pod_advanced_settings_menu.png

Podcast Storage Location

Scroll all the way down until you reach the Podcast Storage Location section and tap on Episode Download Path to enter the folder path. You should probably also click Lock to Current Path as well.

beyond_pod_storage_path.png

Once you change the settings BeyondPod will move the files and restart and at this point it should be storing everything to the SDCard. Now, on to all the other apps in there.

pyLDAvis In org-mode With JQuery

Introduction

In my last post I loaded the pyLDAvis widget by dumping the HTML/Javascript right into the org-mode document. The problem with doing this is that the document has a lot of lines of text in it, which slows down emacs a noticeable amount, making it hard to display one widget, and pretty much impractical to show more than one. So, since Nikola (or maybe bootstrap or one of the other plugins I'm using) is loading JQuery anyway, I'm going to use javascript to add the HTML after it loads from a file.

Imports

Python

datetime is just to show how long things take. In this case the data-set is fairly small so it doesn't take very long, but in other cases it might take a very long time to build the LDA model so I like to time it so I know the next time about how long I should wait.

from datetime import datetime
from pathlib import Path

From PyPi

from sklearn.datasets import fetch_20newsgroups
from sklearn.decomposition import LatentDirichletAllocation
from sklearn.feature_extraction.text import CountVectorizer
import pyLDAvis
import pyLDAvis.sklearn

The Data

I'm going to use the Twenty Newsgroups data-set, not because of anything significant, but because sklearn has a downloader for it so I figured it'd be easiest.

path = Path("~/datasets/newsgroups/").expanduser()
newsgroups = fetch_20newsgroups(data_home=path, subset="train")
print(path)
/home/brunhilde/datasets/newsgroups

The newsgroups.data is a list, so it doesn't have a shape attribute like it would it it were a numpy array.

print("{:,}".format(len(newsgroups.data)))
print("{:.2f}".format(len(newsgroups.data)/18000))
11,314
0.63

The documentation for the fetch_20newsgroups function says that the full dataset has 18,000 entries, so we have about 63% of the full set.

The Vectorizer

I'm going to use sklearn's CountVectorizer to convert the newsgroups documents to arrays of token counts. This is about the visualization, not making an accurate model so I'm going to use the built-in tokenizer. I'm not sure what the fit method is for, but the fit_transform method returns the document-term matrix that we need (each row represents a document, the columns are the tokens, and the cells hold the counts for each token in the document).

started = datetime.now()
vectorizer = CountVectorizer(stop_words="english")
document_term_matrix = vectorizer.fit_transform(newsgroups.data)
print("Elapsed: {}".format(datetime.now() - started))
Elapsed: 0:00:03.033235

The LDA

Now we'll build the Latent Dirichlet Allocation Model.

start = datetime.now()
topics = len(newsgroups.target_names)
lda = LatentDirichletAllocation(topics)
lda.fit(document_term_matrix)
print("Elapsed: {}".format(datetime.now() - start))
Elapsed: 0:02:37.479097

PyLDAvis

Okay so here's where we try and get pyLDAvis into this thing.

Prepare the Data for the Visualization

The Prepared Data

The first step in using pyLDAvis is to create a PreparedData named-tuple using the prepare function.

start = datetime.now()
prepared_data = pyLDAvis.sklearn.prepare(lda, document_term_matrix, vectorizer)
print("Elapsed: {}".format(datetime.now() - start))
Elapsed: 0:00:34.293668

Build the HTML

Now we can create an HTML fragment using the prepared_data function. The output is a string of HTML script, style, and div tags. It adds the entire data-set as a javascript object so the more data you have, the longer the string will be.

div_id = "pyldavis-in-org-mode"
html = pyLDAvis.prepared_data_to_html(prepared_data,
                                      template_type="simple",
                                      visid=div_id)

Export the HTML

Now I'm going to save the html to a file so we can load it later.

slug = "pyldavis-in-org-mode-with-jquery"
posts = Path("../files/posts/")
folder = posts.joinpath(slug)
filename = "pyldavis_fragment.html"
if not folder.is_dir():
    folder.mkdir()

output = folder.joinpath(filename)
output.write_text(html)
assert output.is_file()

So here's where we create the HTML that will be embedded in this post. The JQuery load function puts the content of our saved file into the div. I added the css call because I have my site's font-size set to extra-large, since the Goudy Bookstyle looks too small to me otherwise (I think nice fonts look better when they're big), which causes the buttons in the pyLDAvis widget to overflow out of the header. Under normal circumstances you wouldn't need to do this, but if you do want to do any one-off styling, here's an example of how to do it. Otherwise maybe an update to the style-sheet would be better.

The right-hand box is still messed up, but it's good enough for this example.

print('''#+BEGIN_EXPORT html
<div id="{0}"></div>
<script>
$("#{0}").load("{1}")
$("#{0}-top").css("font-size", "large")
</script>
#+END_EXPORT'''.format(div_id, filename))

Slip Box System Parts List

The Four Parts

A Capture System

This should be paper-based, or at least something that's always there and quick to use.

  • a notebook
  • index cards
  • loose paper
  • napkins…

A Reference System

This is where you put information about your sources. For books and papers Zotero is handy, although once again, the fact that I have to fire up this GUI-based program adds a little bit of overhead. The original system was just another box so I'm going to try something like that. Maybe a sub-folder…

The Slip Box

The original system was a wooden box with A6 paper. I'm using a static site with plain text (org-mode).

Something To Produce a Final Product

The system is aimed at writers, but I'm a computer programmer, and I think it might work with other types of output (like drawing) so it's really just having a way to produce something from your project.

Related Posts

Reference

  • HTTSN - How To Take Smart Notes

Using Your Slip Box

Introduction

This is my re-wording of the Slip Box Method.

The Method

Capture Everything

Write everything down - ideas don't count until they're out of your head and on paper. Writing it down also frees your mind to move on to other things.

Take Notes

Whenever you are taking in someone else's ideas (e.g. reading, listening) take notes.

Make Your Notes Permanent

The initial notes are just temporary inputs, later in the day you need to convert them into some form that has these attributes:

  • They are complete - write them for your future self, don't rely on being able to remember what else is needed to understand the note.
  • They are written in a way that relates to your interests.
  • There is only one idea per note.

Put the Permanent Notes in the Slip Box

  • When you file your note look through the other notes and try and place it behind a related note.
  • Add links to other notes that are related.
  • Add the new note to an entry-point note that holds links to other notes.

Work Bottoms-Up

  • Don't try and come up with a project by "thinking", look through the slip box and let it tell you what you're interested in.
  • If you have an idea for what to do but there isn't enough in the box yet, take more notes.
  • Keep the notes in one folder - don't sort them into sub-categories. This way you can make new associations that you didn't have when you first made the note.

Build Projects From Copies

Copy everything that seems relevant to a project folder on your desktop and see what needs to be filled in and what seems redundant (or maybe just wrong).

Translate Your Notes

Take these fragmented notes and convert them into a coherent argument. If you can't then look to take more notes to fill in what's missing.

Revise

Don't accept the first draft, edit, erase, re-do.

Move On

When you're done with the project, start over with a new one.

Implementation Details

The original method used paper and a wooden box. I really like paper and am tempted to try this, but I don't think doing something that makes me even more if a pack-rat is a good idea. The book (How To Take Smart Notes) recommends a computer program written specifically for this system, but I am a little leery of getting tied into one program, and all these GUI programs are starting to turn me off.

Instead, I'm going to try and use this blog as my slip-box, so, as far as "equipment" goes, this is what I'm going to use:

Flattening out the file-system makes it hard to browse the files, though. I guess less and ls are going to be the main thing I use (and maybe ag and deft). We'll see, I only started reading the book yesterday so I'm still trying to figure this out as I go.

Related Posts

Reference

  • HTTSN - How To Take Smart Notes