Notes from 28/7–1/9

23 min readSep 1, 2019

Unbelievable, it’s been an entire month! But I’ve been keeping up with the newsletters just that I kept forgetting to post on Monday.

Superman Case Study

That was an amazing experience. It was a complete case study of an onboarding experience. It even gave useful snippets that were so powerful.

The Sexual Assault Case That Shook Ancient Rome

Interesting how Cicero himself defended him. At that time, these were disenfranchised women. But she spoke out. Possibly as a political puppet as Cicero suggested too. Yet it’s oddly familiar, how these things are replayed in the modern day.

Spot the psychopath

So perhaps we should think of empathy and psychopaths the same way: they dull their empathic response to others in pain, but they are not naturally insensitive to it.

Ah, well said. So it’s not an affliction, it’s a learned behaviour. A coping mechanism.

A Framework for Moderation

Moreover, Cloudflare is an essential piece of the Facebook and YouTube competitive set: it is hard to argue that Facebook and YouTube should be able to moderate at will because people can go elsewhere if elsewhere does not have the scale to functionally exist.
Second, the nature of the medium means that all Internet companies have to be concerned about the precedent their actions in one country will have in different countries with different laws. One country’s terrorist is another country’s freedom fighter; a third country’s government acting according to the will of the people is a fourth’s tyrannically oppressing the minority. In this case, to drop support for 8chan — a site that was legal — is to admit that the delivery of Cloudflare’s services are up for negotiation.
Third, it is likely that at some point 8chan will come back, thanks to the help of a less scrupulous service, just as the Daily Stormer did when Cloudflare kicked them off two years ago. What, ultimately is the point? In fact, might there be harm, since tracking these sites may end up being more difficult the further underground they go?
This third point is a valid concern, but one I, after long deliberation, ultimately reject. First, convenience matters. The truly committed may find 8chan when and if it pops up again, but there is real value in requiring that level of commitment in the first place, given said commitment is likely nurtured on 8chan itself. Second, I ultimately reject the idea that publishing on the Internet is a right that must be guaranteed by 3rd parties. Stand on the street corner all you like, at least your terrible ideas will be limited by the physical world. The Internet, though, with its inherent ability to broadcast and congregate globally, is a fundamentally more dangerous medium that is by-and-large facilitated by third parties who have rights of their own. Running a website on a cloud service provider means piggy-backing off of your ISP, backbone providers, server providers, etc., and, if you are controversial, services like Cloudflare to protect you. It is magnanimous in a way for Cloudflare to commit to serving everyone, but at the end of the day Cloudflare does have a choice.
None of this is easy. I am firmly in the camp that argues that the Internet is something fundamentally different than what came before, making analog examples less relevant than they seem. The risks and opportunities of the Internet are both different and greater than anything we have experienced previously, and perhaps the biggest mistake we can make is being too sure about what is the right thing to do. Gray is uncomfortable, but it may be the best place to be.

The Agenda — Grassroots Leadership

First, there’s always a better way to do things. Abrashoff’s second insight about change: The more people enjoy the process, the better the results.

Leaders listen without prejudice. From those conversations, I drew up a list of every practice on the ship and divided those practices into non-value-added chores and mission-critical tasks.

Practice discipline withotu formalism. Be low maintenance leader so others don’t have to support you.

Best captains hand out responsibility, not orders.

Successful crews perform with devotion.

True change is permanent. Set up a virtuous cycle that lets people know that their contribution counts.

1. Interview your crew.

2. Don’t stop at SOP.

3. Don’t wait for an SOS to send a message.

4. Cultivate QOL (quality of life).

5. Grassroots leaders aren’t looking for promotions.

AI

TRANSFORMERS FROM SCRATCH

Where do people get the time to write this?

A very comprehensive review of transformers in it’s entirety, starting from the self-attention operation.

The main point of the transformer was to overcome the problems of the previous state-of-the-art architecture, the RNN.

1DConv: In this model, every output vector can be computed in parallel with every other output vector. This makes convolutions much faster. The drawback with convolutions, however, is that they’re severely limited in modeling long range dependencies.

The transformer is an attempt to capture the best of both worlds. They can model dependencies over the whole range of the input sequence just as easily as they can for words that are next to each other.

Moves on to finish the article with a selection of transformers. BERT, GPT2, Transformer-XL, Sparse transformers

Deep Learning Techniques for Text Classification

Crash course on these:

Deep Neural Networks
Recurrent Neural Networks (RNN)
Gated Recurrent Unit (GRU)
Long Short-Term Memory (LSTM)
Convolutional Neural Networks (CNN)
Hierarchical Attention Networks
Recurrent Convolutional Neural Networks (RCNN)
Random Multimodel Deep Learning (RMDL)
Hierarchical Deep Learning for Text (HDLTex)

There’s a bit of code that comes off at one shot, but it’s really just calling packages

The Natural Roots of Artificial Intelligence

“Intelligence is a very general mental capability that, among other things, involves the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly and learn from experience. It is not merely book learning, a narrow academic skill, or test-taking smarts. Rather, it reflects a broader and deeper capability for comprehending our surroundings — “catching on,” “making sense” of things, or “figuring out” what to do.”

General, multiple, behavioural, evolutionary intelligences

“conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves.”

So similar to superintelligence by Bostrom

Advanced Topics in Neural Networks

Transfer Learning
Transfer learning only works if the two datasets that are being analyzed are very similar.

Tuning the learning rate
The only way to achieve better results is to use a dynamic learning rate that tries to leverage the spatial and temporal variations in the optimal learning rate. This is not a trivial task, but several techniques have been developed in order to do this.
First, we can start with a small learning rate and increase it on every batch exponentially. Simultaneously, we can compute the loss function on a validation set. This also works for finding bounds for cyclic learning rates.
Warm restarts are a very simple idea: restart the learning after a specified number of epochs. For example, the learning rate starts at 0.1 initially and decreases exponentially over time. After 30 iterations, the learning rate scheduler resets the learning rate to the same value as epoch 1, and then the learning rate scheduler repeats the same exponentially decay. The best estimates are recorded each time before the learning rate is reset.

It is important to highlight that the total training time of the M snapshots is the same as training a model with a standard schedule. This means that we have a simple method to obtain ensembles of neural networks without any additional training cost.
How to address overfitting

The Bayes error rate is the lowest possible error rate for any classifier of a random outcome and is analogous to the irreducible error. For all intent and purpose, the Bayes error is the minimum error we could obtain with a perfect model stripped of all avoidable bias and variance.
Early stopping is also a popular regularization mechanism, but couples the bias and variance errors.
Orthogonalization is another technique we can use; it aims to decompose the process to adjust neural network performance. It assumes the errors come from different sources and uses a systematic approach to minimize them.
Mc.ai has a wonderful analogy for this of the knobs used to tune radios. The designers of a radio worked hard to make sure that one particular knob controlled one particular aspect of the signal, such as the volume or the frequency. It is much easier to tune to a specific radio frequency and get the sound you want if there aren’t multiple parameters varying simultaneously.
Dropout
It is employed at training time and eliminates the output of some units randomly. This helps to prevent the network from relying on individual neurons too much, which helps to prevent overfitting. Instead, the knowledge is spread across all of the neurons to help obtain a more robust network.
Pruning
Model pruning seeks to induce sparsity in a deep neural network’s various connection matrices, thereby reducing the number of nonzero-valued parameters in the model.
Reducing the number of parameters in a network becomes increasingly important as neural architectures and datasets get larger in order to obtain reasonable execution times of models.

Has lots and lots of references that could really help. I like the examples and explanation.

Data

Introducing the Snail Chart

I don’t see the point of this

Takes up much more space, hard to compare amounts and the claim that it makes it more ‘joyful’ doesn’t seem to work on me.

Data Scientists, The 5 Graph Algorithms that you should know

Clustering
Shortest path
Minimum spanning tree
Pagerank
Centrality measures

Knowing Your Neighbours: Machine Learning on Graphs

For the remainder of this article, we’ll demonstrate how to apply graph machine learning to solve a node classification problem in a homogenous graph. In subsequent articles, we will consider state-of-the-art methods for link prediction and community detection.

Task here is Node classification

Our goal is to train a predictive model to infer the subject of a paper that was hidden from the machine learning algorithm during training. Since subject is categorical with 7 categories, this is a multi-class node classification problem.

Training a 2-layer GCN model (done in this script using our open-source Python library StellarGraph) with 32 output units per layer on the Cora dataset with just 140 training node labels seen by the model results in a considerable boost in classification accuracy when compared to the baseline 2-layer MLP. Accuracy on predicting the subject of a hold-out test set of papers increases to approximately 81% — an improvement of 21% over the MLP that only uses the BoW node features and ignores citation relationships between the papers. This clearly demonstrates that at least for some datasets utilising relationship information in the data can significantly boost performance in a predictive task.

Using the latest advancements in deep learning to predict stock price movements

This covers everything.

Features, statistical checking, feature engineering, GANs, the LSTM to make predictions, learning rate scheduler, hyperparameter optimisation

Clustering & Visualizing Travelers’ Stories with Doc2Vec and WebGL

Doc2Vec to find similar stories

VivaGraphJS to find location-based stories and what are some patterns

Community detection of survey responses based on Pearson correlation coefficient with Neo4j

Now this is extremely powerful and can just be adapted into a jupyter notebook for others to use.

DeepMind’s Latest A.I. Health Breakthrough Has Some Problems

Streams, then, seems typical of DeepMind’s way of working. It offers few overall gains in clinical outcomes, creates anxiety and additional workload for physicians, and was built on the back of deeply controversial access to patients’ data. Whatever Google and DeepMind are planning to do in the United States, they need to overhaul their attitude to the most basic priorities of rights, explanations, and costs to humans, not machines. Those come first, before profit — or rushing to proclaim that A.I. has a central place in the future of medicine.

This article really tried to take them apart. Basically, the model isn’t good. And it violated rights of patients.

How Does Spotify Know You So Well?

Collaborative Filtering models (i.e. the ones that Last.fm originally used), which analyze both your behavior and others’ behaviors. So if you have similar tastes, it is likely you like the same songs as the other person.
Natural Language Processing (NLP) models, which analyze text.
Audio models, which analyze the raw audio tracks themselves.

FastText sentiment analysis for tweets: A straightforward guide.

FastText is an open source NLP library developed by facebook AI and initially released in 2016. Its goal is to provide word embedding and text classificationefficiently. According to their authors, it is often on par with deep learning classifiers in terms of accuracy, and many orders of magnitude faster for training and evaluation. [1]

Our goal is to clean tweets to make them easier to read by a machine. There are many techniques out there to clean text. Most famous ones being lemmatization, stemming and stop words.

Also has a whole list to use for cleaning the data esp slangs, HTML encodings etc.

Once trained, we need to assess how good our model is at sentiment analysis. For this, we can use the two measures Precision and Recall which are the output of fastText functionmodel.test. However, due to the nature of our problem, precision and recall give similar figures and we can focus on precision only.

Development

Software Architecture Guide

architecture was the shared understanding that the expert developers have of the system design. it was more like the decisions you wish you could get right early in a project.

a poor architecture is a major contributor to the growth of cruft — elements of the software that impede the ability of developers to understand the software. High internal quality leads to faster delivery of new features, because there is less cruft to get in the way.

10 cool things you might not know about CSS

Its basically a language for the browser’s rendering engine

Create a revealing animation with Anime.JS

Super super cool

Javascript animation engine!!

Learn DevOps basics with this free 2-hour Docker course

It’s really just installing docker, trying it out and then finally kubernetes

An overview of Progressive Web App development

Progressive app development is strongly patronized by Google Inc., which believes PWA development to be the future of software. According to Google articles, with progressive web apps, AliExpress has boosted the conversion rate for its new users by 104% and increased average time spent per session by about 74%.

PWA development is a set of optimal software development practices aimed at making a web application function similarly to a mobile or desktop app. Similarly to a mobile application, PWAs send push notifications and have an icon on the home screen. At the same time, progressive web applications are simpler and faster than traditional mobile apps, and they can be shared through a URL.

Making Apache Spark Effortless for All of Uber

It’s quite an engineering feet to put together all these open sources tools into a single architecture. All this does is really just pulling in data, cleaning it for applications. Yet it’s so complex. You’ve got to handle the resource provision, the IDEs, how to do batching etc

The responsibility of a developer with regards to climate change

Can your digital bullshit run on one small machine? I bet It can. You need a failsafe deployment? Use two small machines
Aim for the most efficient solution
You should not produce software and never run anything that doesn’t make sense. Don’t work for any company just because you need the money. The market is very much in favor of us developers right now. Use that to make a difference.
Learn new languages that have super-efficient runtimes, support zero-cost abstractions etc etc, like Rust, Go, Pony etc etc and avoid interpreted bullshit like python or ruby that has been optimized for minimizing developer time.Development time shouldn’t be the main concern anymore, resource efficiency should. If you find something that makes you productive and produces highly efficient programs, use it. There is no excuse! And if you are not productive at the start, hang in there and shape the tools to become productive.
Learn about low level stuff like cache locality, SIMD, GPU programming etc etc Knowing your CPU gives you tons of stuff that can make your program run orders of magnitudes faster, look at e.g. simdjson. Performance equals saved CPU cycles.

The Software Developer’s Guide To Career Ownership

Taking ownership of your career means that you realize your career is an organic thing that you can either nurture and grow, ignore or even damage.

Reputation = How much you stand-out + competence.
Tip: Show an interest in your manager. Make their job easier by doing awesome work. And, ask about how your work is improving the company.
One-on-one meetings are a great time to ask about these things.

Get On A High Visibility Project

This isn’t so you can become famous. It’s so that you can:

Gain experience fast
Become known in your org as someone who gets things done well

You need to have real-life experience and learn from real-world situations.

But, you also need to push your knowledge about what tools and techniques are available to solve problems.

Here are some general tips for learning and growing in your technical skills:

Figure out what tech stack you want to be skilled with and learn it in-depth
Read every day! (This will help you become a better reader and just help you learn every day)
Map out your skills and figure out which ones you want to dig into and which ones you want to learn the fundamentals about
For skills where you want to know the fundamentals, the most important thing to understand is what problem(s) does it solve?
For skills that you want to dig into, consider mentoring or high-quality training material (think Pluralsight)
If your current job isn’t challenging, consider moving on to something more challenging

What you need to know about DNS

One problem this poses for DNS is that there isn’t any verification of the authenticity of the name server when a response is received. Thus a hacker can send malicious responses to a computer’s DNS query and trick the computer into thinking that it is the real response from the DNS nameserver. In other words, when the computer asks, “what is the IP address for www.chase.com?” the hacker will respond (before the DNS server can) with the IP address for the hacker’s malicious site . Then when the site loads, it looks just like the chase.com website, but is actually controlled by the hacker.

Your computer essentially recursively checks for the IP address in higher and higher level servers.

9 Useful Browser Extensions for Developers — 2020 edition

Browser extensions are really interesting ideas to play with here. It’s just easy to use. One point is that these extensions assumed certain behaviours of developers.

JavaScript Fundamentals: Understanding APIs

Broad overview of browser and 3rd party APIs. Briefly explains how to use them and what they do.

I Wrote a Script to WhatsApp My Parents Every Morning in Just 20 Lines of Python Code

Twilio + AWS Lambda

Really simple way to create a text messaging system

Let’s build a full stack MongoDB, React, Node and Express (MERN) app

Basically an app that performs CRUD functions, just for practice.

The Google Cloud Developer’s Cheat Sheet

SOOO USEFUL

The Next Feature Fallacy: The fallacy that the next new feature will suddenly make people use your product

Let’s introduce the concept of an engagement wall, which exists at the moment that your product asks the user to deeply invest in their product usage, where “behind the wall” means that the feature can only be experienced once the users buys into a product, and engages.

Life Optimisation

How To Read Academic Content Once and Remember it Forever

Your primary objective when approaching your learning materials is to extract all of the important concepts and facts. You’ll need to process that information in very specific ways in order to create an infrastructure of knowledge that will provide the foundation for later higher-order thinking processes.

When you consume academic content, your mission is to parse, comprehend, and extract

Spaced-repetition flashcards for concepts and facts — use that to link directly into the source content that you need
If there’s a concept you don’t get, Google and look for other ways to learn that concept

Actively consume and comprehend the material first. Record the key concepts and facts that you want to remember by creating spaced-repetition flashcards. Then test and build your recall ability of what you’ve learned with flashcards!

some meta questions to ask:

How would I briefly summarize this? What are the key concepts?
Does this remind me of anything that I already know in this domain of knowledge?
Does this remind me of anything in another unrelated domain of knowledge?
What other things do I know that support the veracity of this concept?
Can I think of anything else I know that contradicts this concept?
Do I know any concrete or practical examples that illustrate or illuminate this concept?
How challenging is it for me to comprehend this?
If this is a WHAT, can I explain the WHY? (or vice versa, as applicable)
Why is this so important to know?
Do I find this interesting? Why is that the case, or why not?
Do I find this surprising? Why is that or why is that not the case?
How can I apply this in the real world?

Staying Connected is Key to Your Startup’s Survival — Here’s How to Nail Internal Comms

When teams are in sync, it fortifies every aspect of company and product-building

Employees really should be your number one audience, especially as you scale.

Enter Open Houses: smaller, informal meetings that anyone could announce and set up via the company’s internal messaging platform. “There’s so much going on at SoundCloud. Email and documentation aren’t always the best ways to engage around a topic that is dynamic and exciting,” Noël says. When a company is small, it’s easy to get information informally, over a meal or during a hallway chat. But as a team grows, these lines of communication can break down. “Open Houses provide a way for people to stay informed about all of the things they choose to care about.”

Any kind of transition is bound to make people uneasy, because change is frightening.

Your internal communication strategy is your opportunity to take control of the narrative, right from the beginning

Every productivity thought I’ve ever had, as concisely as possible

Having no clear idea what to do next increases the probability that you won’t feel like following all the rules you came up with massively. The only solution I know is to avoid working from home as much as you can.

Tabs have a tendency to blow up. However, there’s a natural upper limit for how much the can blow up, since at some point they overflow and you no longer have access to the rightmost tabs. This is so frustrating that we can naturally turn this to our advantage. So, create two soft rules:

only close tabs from the left-hand side
only open tabs in the end (but can close just opened tabs).

How to Provide Great Feedback When You’re Not In Charge

APPRECIATION is expression of gratitude or approval of another’s effort. It is an expression of emotion, designed to meet an emotional need.
ADVICE (or COACHING) consists of suggestions about particular behavior that should be repeated or changed. It focuses on the performance, rather than judging the person. What worked well, what could be done differently.
EVALUATION is ranking the subject’s performance in relation to that of others or against an explicit or implicit set of standards.
Be willing to take the first fall. CReate an environment where feedback is shared in a helpful and useful way.

How To Introduce People: Two Rules For Better Networking

Every introudction is also a recommendation.
Check with both parties first before introducing

What, that’s it?

AI Masters Degree

Reduce the search space
Fix your environment
Set the system up so you always win — go for different goals if you can’t solve problems yet
Sometimes do nothing — focused thinking, diffused thinking. Work on both
Embrace the suck
The 3 year old principle — see everything as play
Sleep

The 10x Engineer

We believe that 10x engineers are not only people that can produce 10x faster than most people, even more, those are people that can make a whole team 10x better by having a positive influence on anyone that they work with.

Independence, planning, reuse, infrastructure mindset, master your domain, curiosity, no boundaries, responsibility, ownership, communication, teamwork, keep it simple, prioritising, time management

Programming

2020 and Beyond Programming Trend Predictions

Rust will go mainstream Learn here
GraphQL adoption will grow
Web Apps
WASM
React is the way to go
Javascript in general is the way to go

142 Resources for Mastering Coding Interviews

Packed with goodies

The Ultimate Strategy to Preparing for a Coding Interview

Well, I wasn’t solving coding problems but practicing to map problems onto problems that I’d already solved. I used to read a problem and spend a few minutes to map it to a similar problem I’d seen before. If I could map it, I’d focus only on the different constraints this problem had compared to the parent problem.

Includes some coding examples that can help with coding interviews if necessary.

Python is eating the world: How one developer’s side project became the hottest programming language on the planet

So how did Python leapfrog its erstwhile rival, and how to explain the two languages’ vastly different fortunes? Van Rossum believes it has something to do with how easy it is to maintain a code base once it grows beyond a certain size. “People’s experience was that for a 10-line script, Perl is perfect,” he says. “But if you have 500 lines of mainline code and a few thousand lines of library, it requires an enormous amount of discipline to make that code maintainable in Perl. While in Python, even if you don’t have all that much discipline, the code will still be fairly readable and fairly maintainable.”

Usability

“In the past, it had always been clear that if there were a decision to be made about a change in the language or an improved feature, a whole bunch of core developers would discuss the pros and cons of the thing. Either a clear consensus would appear or, if it was not so clear, I would mull it over in my head and decide one way or another. With PEP572, even though it was clearly controversial, I chose ‘Yes, I want to do this’, and people didn’t agree to disagree.

Governance

Product Management

Shade

Interesting way to market a product. Which is basically a landing page

Probably built from normal, open source frontend libraries

Interesting way to package products

Adam Nash’s Guide to Product Planning: 3 Feature Buckets

Interesting way to use Coda

Managing and Developing Product Managers

It’s their vision, you just buy into it.

Investors don’t mettle in a portfolio company’s operations

Focus on helping to support their blind spots

They should have more to gain in success than you do

They should have more to lose in failure

PMs develop best when shipping Whole Customer Experiences

When PM career development is centred on solving whole customer experiences:

i. the PM can always take a product all the way to market

ii. the PM can operate autonomously (because of the above), and thus it reinforces the investor<>entrepreneur model

iii. It’s easy to explain to a PM how their careers will grow — as they progress they will solve the whole problem for product areas of ever-increasing scope (the definition of which we’ll cover next)

Finding the right scope for a PM

Customers Impacted — self explanatory
Revenue/Cost Impact — self explanatory
Strategic Importance — how mission critical is the success of this project/product to the future of the company?
Operational Complexity — does this product fit into the existing operational systems of the company? Can the existing support team manage issues or do we need to create new operations?
New Company Domain — is this a new area for the company? For example, is it a consumer product now dabbling in enterprise?
Resourced Required — what percentage of the company’s people are allocated to this project or product area?

Assessing PMs for Performance

A. Product Performance

B. How well they work with other people

C. Decision making ability

D. Strategic thinking

The Past and Future of Product Management

I believe that the future of product management rests upon our ability to embrace human complexity, in both the processes we implement to build products and the data we consult to understand customers.
I believe that product management is a fundamentally supportive and facilitative role, not a “visionary” role. (No Steve Jobses need apply.)
I believe that the distinction between “hard skills” and “soft skills” is largely counterproductive, and that the best product managers possess the connective skills needed to bridge diverse perspectives and roles within an organization.
I believe that many of the traditional “profiles” of product managers are narrow and misguided, and there is enormous untapped potential for great product managers beyond “engineers who like design” or “UX designers who can code.”

Original Agile

It is somewhat ironic, then, that this set of values was codified into a highly prescriptive and static process…

… which then became an ecosystem of increasingly specific and minutely differentiated processes and tools…

… each of which required its own comprehensive documentation and set of rules.

My friend Mike Dewar, who is a data scientist at the New York Times, wrote an excellent piece about this very thing, arguing that recommendation engines are not about maximizing metrics, but rather about designing experiences for people.

PM candidates who are exactly like engineers are often the most well-received by engineering teams during the job interview process. And why wouldn’t they be? They often have similar experiences, similar interests, and similar expertise. The developers have a lot to talk to them about. And the hiring manager, who is afraid of alienating the engineers by bringing on a “non-technical” product manager, breathes a sigh of relief.

The differentiating factor between success and failure is not being “technical enough,” but rather being adept at making connections between different sets of values and expertise.

Great PMs can make connections between seemingly disparate experiences, value systems and skill sets.

Dashblock

This is pretty amazing

4 skills growth product managers need to succeed

Mindset: scientific. curious, skeptical and analytical.

Agile: speed and maneuvering

Diplomatic: able to process many different and divergent viewpoints and opinions

Communicates: keeps people in the loop and build relationships

Security

We Need To Dump Our Role-based Security World!

Well, everyone knows that we should be encrypting data at its core level (or in the application layer), and not relying on tunnels or single encryption keys for our protection. In this way, we can embed encryption into the actual data and then define the actual access rights (and without relying on domain/operating system rights). Few people still trust the security of user names and passwords to properly protect data, but we still blindly use it for our accesses to data.

CP-ABE to encrypt data according to access rights.