Books and peer-reviewed articles read in 2020

James Clear: Atomic Habits

An Easy & Proven Way to Build Good Habits & Break Bad Ones.

• Habits form based on frequency, not time
• Identity based habits focus on who you want to BECOME. Your identity emerges out of your habits. Every action is a vote for the type of person you wish to become. 4 laws
• Make it obvious.
  • e.g. pointing and calling (surface consequences)
  • Habit stacking (attaching habits to existing ones)
• Make it attractive
  • Approval, respect and praise - from the close family; “many”, and the powerful.
• Make it easy
  • To establish a new habit, start small, engage in the relevant activity for only two minutes. This “showing up” helps to “ritualize the beginning of a process.”
  • “commitment devices” (e.g. pre-pay)
• Make it satisfying
  • Choose a reward that fortifies the habit stacks that fit your identity
  • Use a log to gauge your progress and maintain focus
  • Habit contracts (make not delivering expensive)
  • Work on the edge of your abilities

Innovator’s DNA

Innovative companies are almost always led by innovative leaders.

Two-time CEO at Procter & Gamble, declared, “innovation is the central job of every leader—business unit managers, functional leaders, and the CEO.”

Book and article

Insights:

A critical insight from our research is that one’s ability to generate innovative ideas is not merely a function of the mind, but also a function of behaviors.

• associating
• questioning
• observing
• networking
• experimenting

We were intrigued to learn that at most companies, top executives do not feel personally responsible for coming up with strategic innovations. Rather, they feel responsible for facilitating the innovation process. In stark contrast, senior executives of the most innovative companies—a mere 15% in our study—don’t delegate creative work. They do it themselves.

Discovery Skill 1: Associating

Medici family brought together people from a wide range of disciplines—sculptors, scientists, poets, philosophers, painters, and architects. As these individuals connected, new ideas blossomed at the intersections of their respective fields, thereby spawning the Renaissance, one of the most inventive eras in history.

Based on a lifetime exploring new and unrelated things

“It’s very helpful to cross-fertilize ideas from different industries,” Tesla’s Elon Musk told us. “Sometimes a solution from one industry can apply to a different industry. That can be really powerful.”

IDEO describes this person as “T-shaped,” because the person holds deep expertise in one knowledge area but actively acquires knowledge broadly across different knowledge areas.

In addition to hitting the shower, Greene gets some of her best new associations when sailing solo (which she has done since childhood).

Discovery Skill 2: Questioning

“question the unquestionable.” Try spending 15 to 30 minutes each day writing down 10 new questions that challenge the status quo in your company or industry.

Ask “Why?” and “Why not?” and “What if?”

“Why are we still loading and upgrading software the way we’ve been doing all this time when we can now do it over the internet?”

Imagine opposites.

Innovative thinkers have “the capacity to hold two diametrically opposing ideas in their heads.” He explains, “Without panicking or simply settling for one alternative or the other, they’re able to produce a synthesis that is superior to either opposing idea.”

Embrace constraints. Ironically, great questions actively impose constraints on our thinking and serve as a catalyst for out-of-the-box insights.

Examine sunk-cost constraints by asking: “What if you had not already hired this person, installed this equipment, implemented this process, bought this business, or pursued this strategy? Would you do the same thing you are doing today?”

Dicovery Skill 3: Observing

In observing others, they act like anthropologists and social scientists.

Discovery Skill 4: Experimenting

“If we can get processes decentralized so that we can do a lot of experiments without it being very costly, we’ll get a lot more innovation.”

One of the most powerful experiments innovators can engage in is living and working overseas. Our research revealed that the more countries a person has lived in, the more likely he or she is to leverage that experience to deliver innovative products, processes, or businesses.

Discovery Skill 5: Networking

innovative entrepreneurs go out of their way to meet people with different kinds of ideas and perspectives to extend their own knowledge domains.

Paper 12 - Why employee experiences fall short (HBR article)

https://hbr.org/2020/07/why-employee-experience-initiatives-fall-short

Idea: HR leaders ask employees to reflect on how the programs have improved their work lives and encourage them to post their stories on the road map and share them in team meetings, off-sites, internal newsletters, and other forums. As companies emerge from the current crisis, leaders might seek ways to highlight examples of how they supported their employees, such as by continuing to pay sidelined workers or providing extended sick leave.

Phil Knight: Shoedog. Nike

Source

Paper 11 - Deep Learning and Time Series-to-Image Encoding for Financial Forecasting

Source

The idea is to convert financial data (time series) into images, and then using ensembles of CNNs to predict trading actions. I’ve never heard about “Gramian Angular Fields” either (see e.g. here , so I did learn something here!

Paper 10 - Welcome to the 21st Century: How To Plan For The Post-Covid Future

Source

We’re at an inflection point (better acknowledge it). No way we’ll get back to “normal”

• We’re surprised when “ignored knowns” hit us.
• So let’s do scenario planning, open our mind!
• A few examples
  • what if disease supplants terrorism as our daily devil?
  • as the death knell to privacy?
  • end of health insurance as we know it in the US?
  • emergence of “remote native” firms?
  • kids without being socialized in schools?

  • distrust in governments effectiveness?

  • “A bunch of industries look like candidates to get a decade of inevitability in a week.”. Are business models sustainable under the new conditions?
  • Cloud computing, streaming performance, new sources of big data and better tools to manage it, are in higher demand than ever.
  • You’re looking for is a “robust strategy,” one that will hold up well against a range of possible outcomes.
  • Encouraging people to borrow money so they can buy things they don’t need is not robust.
  • There are only a few projects that are large enough to warrant the amount of stimulus needed to cope with economic emergencies at this grand scale. Building the infrastructure for addressing climate change is just such a project

Paper 9 - Batch normalization - The Greatest Breakthrough in Deep Learning

Source I know this is a cop-out and this is a rather quick read - but - I really didn’t know about batch normalization until I read this article, so given this was a LOOONG week I still give myself “minor credit” for trying to keep up

Paper 8 - Language models are Few-Shot learners

Source

GPT model was scaled up to … drum rolls … 175bn parameters (10x of MS’s Turing team), giving the model similar performance characteristics with few shot learning that previously you needed to fine tune on. Also has good overview of zero, vs one vs. few shot approaches. Wow - we really live in the future!

Paper 7 - The changing world order / Ray Dalio

Branching out in a different direction

Ray Dalio has impressed me greatly, not only seen through the lens of financial rewards (as a billionaire he must have done a few things right), but rather how seriously he takes the scientific method in his day job as a macro investor. The TL;DR on his approach is “develop archetypes of big things (cycles) happening in the world, break them down into “if then” statements, and then place bets and refine the approach as instances of those archetypes occur” (e.g. great depression). Secondarily, his perspectives on an increasing polarity in the US (eg. wealth gap), and how this polarity increases in crises (like now, high unemployment while stock market is doing fine for those with capital) are worth understanding.

Apparently quantitative easing pushes asset prices up, which benefits the wealthy who hold more of those assets.

Paper 6 - World of Bits. An environment to train agents for web tasks

I wasn’t aware that there was a publicly available Open AI Gym style environment for web tasks - “an open domain platform for web-based agents”.

Resource

Github code & docker: https://github.com/openai/universe

The key here is that the environment provided simulates a website without actually hitting Delta.com, which allows at scale training via reinforcement learning.

Paper 5 - More research around BERT - Named Entity Recognition

Source
While this is not a “peer reviewed” article in the technical sense, this post still provided a great walk-through of how to do entity recognition in practice.

Paper 4- Exploring the Limits of Transfer Learning (T5)

Source: It’s massive at 53 pages, from Google

Includes code, training data, and pre-trained models

• Insights:
• Presents a unified framework that converts every language problem into a text-to-text format. It obtained comparable performance to task-specific architectures (achieves state-of-the-art with scale)
• Original encoder-decoder architecture works well with text-to-text approach
• Introduces new text data set “Colossal Clean Crawled Corpus”. Large diverse language data set (used only once during pre-training) works best
• Training a smaller model on more data was often outperformed by training a larger model for fewer steps
• Larger modes == better (sucks for on-device scenarios)

Paper 3-Image segmentation in 2020: Architectures, Losses, Datasets, and Frameworks

Source

A good overview of various image segmentation algorithms (U-Net. FastFCN, Gated-SCNN, DeepLab. Mask R-CNN; and also available datasets such as Coco

Paper 2-HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training

Source

• Why it matters: For RPA, we need to reason over temporal sequences. Well, this paper captures multi-modal & temporal. While Oscar was about image & text pairs, HERO is about video + language pre-training (along the lines of VideoBERT)

• Key points: Hierarchical encoding

  • Capture GLOBAL video context (temporal transformer) together with LOCAL textual content (multi-modal fusion)
  • Trained on TV show data sets; able to e.g. learns complex social scenes.
  • Two steps: (1) Fuse subtitle sentence with video frame (cross-modal transformer) and (2) temporal transformer (sequentially contextualized embeddings) –> clip-level temporal context

Paper 1-An application of Vision-language pretraining (VLP)

Arxiv paper

• Why it matters: Cross-modal training of e.g. text and image is a promising avenue for e.g. our RPA efforts. Key insight of the paper: Instead of simply concatenating image region features and text features the authors use object tags detected in images as anchor points to significantly ease the learning of alignments.

• Key points Based on Self-attention Transformers (BERT style). Based on singular embeddings (i.e. single vector which contains word embeddings and visual region features) For this specific method, input to training is (1) TEXT - a description (2) TEXT - tags extracted from image (3) PICTURE - image regions Uses Faster R-CNN to extract (1) tags (2) position-sensitive visual region feature detector Two training objectives: (a) recover masked language tokens (across description & tags) (2) swap out a tag and classify if data row is “polluted” or “not” (same as BERT where the question is “does one sentence follow another one”) Beats state-of-the-art for many downstream tasks (e.g. caption generation) Reproducible results: https://github.com/microsoft/Oscar

Chris Fussell, C. W. Goodyear: One Mission: How Leaders Build a Team of Teams

Source

There are benefits to driving 700 miles in a day - a full audio book!

TL;DR - you can combine the agility of teams with the benefits of vertically oriented organizations (bureaucracy) by fostering horizontal connections, a shared narrative, and forums to facilitate these.

The danger of highly cohesive sub-groups & vertical organization

• Echo chambers and “inter-tribal conflicts”
• Local success does not mean Global maxima (team win, but org overall does not achieve mission)
• Vertically oriented reviews make it hard to detect horizontal misalignment

Solution:

• shared mission across the organization. Leaders constantly reinforce the group’s unification process by sharing stories that exemplified unity.
• Shared consciousness and understanding.
• Horizontal connections. E.g. to gain the cooperation of critical partners or external stakeholders you can identify specific people to serve as liaisons.

How to get there:

• Frequent coordination: the author uses the example of Operations & Intelligence forum which runs daily (!) and brings myriad participants together from around the globe. Participants become “boundary connectors” and are being aligned around a shared narrative.
• Author frequently talks about “Contextualization” – forums to connect and provide shared understanding
• Avoids managers as information pumps getting overloaded
• The power of being a hub in the organization (need to work on that!) – horizontal links
• Allow new team leaders to learn the informal org structure before they are allowed to act (e.g. “No fly zone” for 2 months)

Talks about the importance of the environment:

• How quickly does the environment change – do the processes in the org allow quick enough response? Are the systems managers dealing with complicated or complex (need different strategies!)
• Rhythm of business needs to adapt to the speed of the changing environment

Patty McCord: Powerful. Building a culture of Freedom and Responsibility

• Minimize rules, processes, instructions
• Communicate - how do their jobs contribute to their own goals, team’s mission, overall mission of the company
• Provide context - things the org is struggling with, where it’s stumbling, communicate broadly daily
• Radical honesty
• Fierce debate
• Sports team metaphor, not family
• Perks don’t motivate
• You are hired to do a job, if the job is not necessary any more, you should go

Kevin Scott: Reprogramming the American Dream

Thoughts about platforms:

• most powerful platforms have self-improvment feedback loops built in (they get better, increasingly faster, until they hit some sort of plateau), examples: electricity; PC; internet; AI
• Engineering those interfaces to the to the physical world tends to be the most time consuming part of applying AI to a task.

Cal Newport: Deep Work

Rules for focused success in a distracted world

• Professional work == intense focus + concentration.
• Time spent on task X intensity of focus.
• Evaluate habits and actions: structure your time to protect the attention you need for deep work (daily units, or withdraw periodically)
• ‘Embrace boredom’, ‘quit social media’ and ‘drain the shallows’ of your life.

Ray Dalio: Principles

Website with details

Miguel Grinberg: Flask Web Development, 2nd Edition

Learned how to build a website with Flask and Python. Sample code is here on github

Angela Lee Duckworth - “Grit” the power of passion and perseverance

“Talent” is the measure of how fast your “skill” advances when you apply yourself to your chosen task.

• “talent x effort = skill.”
• “skill x effort = achievement.”

You must persevere to build your skills - life demands nonstop effort. More grit leads to more grit.

Power of deliberate practice: “stretch” objective <-> thoughtful replication with continual “refinement”

Steven Johnson - “Farsighted”

Book summary.

Overall usefulness: 4 stars

Key premise: Outcomes with the highest uncertainty are often the ones we care about most. So we should use tools to help us decide well.

Setting the context

• Good news: Complex decisions often harbor undiscovered options
• Bad news: Especially low-ability individuals overestimate their abilities greatly
• Traps:
  • People are generally unwilling to give up things they think belong to them
  • When making decisions, focusing on an anchor variable only leads to bad decisions

How-to decide well

(1) Create the map of variables (2) predict outcomes (3) make decisions against our objectives

1 - Mapping

• Tools: storytelling. They don’t give us ready solutions, but allow us to practice
• To derive the most useful information from multiple sources of evidence you should always try to make these sources independent of each other before an issue is discussed, all members of the committee should be asked to write a very brief summary of their position (groups generally focus on shared information)
• Avoid the cognitively central in-group
• Just the presence of diverse group participants makes a difference
• Introduce expert roles to get full-spectrum thinking
• Ask people for their level of certainty (allows better assessment of where things really are)
• Seek out contradictory evidence explicitly
• Consider more alternatives – when you find yourself in a “whether-or-not” situation, you should reframe it into a “which option” question that gives you more available paths

2 - Predicting

• Randomized controlled trials; ensemble forecasts
• Scenario planning
  • One where things get better
  • One where things get worse
  • One where things get weird
  • Resist the fallacy of interpolation
  • Construct multiple stories
• Pre-mortems – the plan has failed. Explain why!
• Red-teams – devil’s advocate

3 - Deciding

• Tools: Bad events table of values model

Concepts

• Cone of uncertainty
• Full-spectrum vs. narrow-band thinking
• Randomized controlled trials
• Ensemble forecasts

Other interesting observations

• Progress of science can be measured by the speed at which we create things to test