Sunday, February 25, 2018

Zombie projects and how to kill them

source

“You will never find them,” said a senior leader in a multibillion-dollar IT company.

The “them” the leader was referring to were zombie projects: the nefarious enemies of well-intentioned innovation efforts around the globe. Zombies are projects that, for any number of reasons, fail to fulfill their promise and yet keep shuffling along, sucking up resources without any real hope of having a meaningful impact on the company’s strategy or revenue prospects.

We had suggested that at least one reason why the company was struggling to successfully commercialize innovative ideas was that zombies were draining its resources and clogging its pipeline. The leader was skeptical.

He thought we wouldn’t find any given the company’s highly rigorous planning process. Every year scores of people spent months reviewing recent performance and sanity-checking future projections. Every project went under the proverbial microscope. So how could a zombie project possibly exist?

A zombie project spawns in predictable ways. The project certainly makes sense when first sanctioned by leadership. Its financial projections, while always uncertain, look reasonable. Market assumptions seem plausible. The development timeline looks achievable.

But somewhere along the line, something happens. The technology doesn’t quite work as planned. A competitor does something unanticipated. A key partner decides not to participate. Customers react in an unexpected way.

Project team members know that what’s happened isn’t good, but it’s hard for them to acknowledge when a project has come off the rails. Psychologists have pointed out how we suffer from confirmation bias, paying more attention to the things we expect and ignoring the things we don’t. And even when we’re aware of setbacks, we’re prone to using the affect heuristic — when we believe in something, we play up good news and ignore bad news.

At some point the data do become overwhelming, and if you gave team members truth serum, they’d admit that the project will never contribute meaningfully to the company’s financial and strategic goals. But since in most companies reward systems carry strong penalties for failing to meet commitments, people hesitate to raise their hands and say “Our project is one of those.” It just looks smarter to find ways to stay alive.

We had spent enough time with our IT company to know how skillfully project leaders could subvert the disciplines of the budgeting process to keep their zombies shuffling along. One recipe for survival: project big revenue numbers five years in the future but ask for only modest investment in the near term. In the next budget cycle, repeat the process so that projected revenue always stays safely beyond the planning process’s two-year horizon. As long as the team successfully manages its costs, everything’s fine, since there’s essentially no penalty for perpetually projecting, but never hitting, long-term targets.

Every budgeting system has its quirks, and innovators in survival mode will skillfully find and exploit them. To fight against these challenges we proposed a “zombie amnesty” — a period during which people can come clean, put their projects up for consideration, and suffer no repercussions if a project is terminated. The critical point of the amnesty is not to lay people off to cut costs but rather to allow the company to invest in new growth by redeploying them to more promising projects.
When we evaluated three dozen efforts for this IT company using realistic projections of possible revenues, we found 20% of them were zombies that didn’t warrant continued investment. Shutting those projects down without penalty would free up enough funding to support two years of more strategic innovation activities.

In a December 2014 HBR article, we argued that these kinds of zombie amnesties are a vital component of a systematic approach to innovation. But they aren’t easy to pull off. Based on our work and the work of like-minded academics — most notably Rita Gunther McGrath of Columbia University (a certified zombie killer if ever one existed) — we’ve identified six keys to doing it successfully:
  1. Use simple, transparent, predetermined criteria. Shutting a project down can be very emotional. Setting and sharing a shortlist of criteria before the process begins helps participants view the process as rational. At the most basic level, we always ask three questions about an idea: Is there a real market need? Can we fulfill that need better than current and potential competitors? Can we meet our financial objectives? Whatever the criteria, remember they are guidelines, not rules. Final decisions will always require some degree of subjective judgment.
  2. Involve outsiders. Parents will attest to how hard it is to be objective about something you’ve played a part in conceiving. An uninvolved outsider — someone from a different division or from the outside entirely — can bring important impartiality to the process.
  3. Codify lessons learned along the way. McGrath teaches that any time a company innovates, two good things can happen. The idea is successfully commercialized (clearly good), or — even if it is not — you learn something that sets you up for future success. Hold action-after reviews to capture lessons learned and create a living database to store and share those lessons. As research shows that “knowledge gained from failures [is] often instrumental in achieving subsequent successes,” investing to capture and spread knowledge from your zombie projects maximizes the return on those investments.
  4. Expand the definition of success. Executives at large companies often fret about how to match the upside potential enjoyed by start-up entrepreneurs. They should spend far more time worrying about what happens to innovators that work on projects that don’t succeed commercially. After all, when taking well-thought-out risks carries the risk of punishment, it’s no surprise that people hesitate to take any risks. Any time you innovate, future success is unknown. Therefore, learning that an idea is not viable is a successful outcome, as long as those lessons are learned in a reasonably resource-efficient way. Pat team members on the back when they’ve given you that precious gift.
  5. Communicate widely. This might sound counterintuitive, but broadcasting commercial failures widely encourages future efforts, because innovation happens most naturally at companies that “dare to try.” That actually is the name of an award given by the Tata Group, India’s leading conglomerate. The award “recognises and rewards [the] most novel, daring, and seriously attempted ideas that did not achieve the desired results.” Shining a spotlight on these kinds of efforts naturally makes it safer for people to push the innovation boundaries. After all, if you don’t dare to try, how can you hope to succeed?
  6. Provide closure. This idea is ripped straight from McGrath’s excellent 2011 HBR article, “Failing by Design”: “Have a symbolic event—a wake, a play, a memorial—to give people closure.”
The Finnish mobile gaming company SuperCell, which was valued at $3 billion only three years after its founding, demonstrates the power of following these disciplines. At SuperCell, success is celebrated with beer, failure with champagne. Mistakes are addressed with brutal honesty, as when after a year of development and investment the company decided to scupper a multi-platform approach that fell short of its development targets. By decisively killing a potential zombie project and yet celebrating the good work of the team, SuperCell allowed its members to shift their focus to a better idea. In this case, they went on to develop the massively successful Clash of Clans game.


Almost every company has more resources than it realizes. Find and put the zombies down, reallocate resources to your most promising projects, and you will suddenly find your innovation efforts getting better and bigger faster.

Best Python Package layout

source

Packaging a python library

Sun 25 May 2014

Note
This is about packaging libraries, not applications.

All the advice here is implemented in a project template (with full support for C extensions): cookiecutter-pylibrary (introduction).
I think the packaging best practices should be revisited, there are lots of good tools now-days that are either unused or underused. It's generally a good thing to re-evaluate best practices all the time.
I assume here that your package is to be tested on multiple Python versions, with different combinations of dependency versions, settings etc.
And few principles that I like to follow when packaging:
  • If there's a tool that can help with testing use it. Don't waste time building a custom test runner if you can just use py.test or nose. They come with a large ecosystem of plugins that can improve your testing.
  • When possible, prevent issues early. This is mostly a matter of strictness and exhaustive testing. Design things to prevent common mistakes.
  • Collect all the coverage data. Record it. Identify regressions.
  • Test all the possible configurations.

The structure

This is fairly important, everything revolves around this. I prefer this sort of layout:
├─ src
│  └─ packagename
│     ├─ __init__.py
│     └─ ...
├─ tests
│  └─ ...
└─ setup.py
The src directory is a better approach because:
  • You get import parity. The current directory is implicitly included in sys.path; but not so when installing & importing from site-packages. Users will never have the same current working directory as you do.
    This constraint has beneficial implications in both testing and packaging:
    • You will be forced to test the installed code (e.g.: by installing in a virtualenv). This will ensure that the deployed code works (it's packaged correctly) - otherwise your tests will fail. Early. Before you can publish a broken distribution.
    • You will be forced to install the distribution. If you ever uploaded a distribution on PyPI with missing modules or broken dependencies it's because you didn't test the installation. Just beeing able to successfuly build the sdist doesn't guarantee it will actually install!
  • It prevents you from readily importing your code in the setup.py script. This is a bad practice because it will always blow up if importing the main package or module triggers additional imports for dependencies (which may not be available [5]). Best to not make it possible in the first place.
  • Simpler packaging code and manifest. It makes manifests very simple to write (e.g.: you package a Django app that has templates or static files). Also, zero fuss for large libraries that have multiple packages. Clear separation of code being packaged and code doing the packaging.
    Without src writting a MANIFEST.in is tricky [6]. If your manifest is broken your tests will fail. It's much easier with a src directory: just add graft src in MANIFEST.in.
    Publishing a broken package to PyPI is not fun.
  • Without src you get messy editable installs ("setup.py develop" or "pip install -e"). Having no separation (no src dir) will force setuptools to put your project's root on sys.path - with all the junk in it (e.g.: setup.py and other test or configuration scripts will unwittingly become importable).
  • There are better tools. You don't need to deal with installing packages just to run the tests anymore. Just use tox - it will install the package for you [2] automatically, zero fuss, zero friction.
  • Less chance for user mistakes - they will happen - assume nothing!
  • Less chance for tools to mixup code with non-code.
Another way to put it, flat is better than nested [*] - but not for data. A file-system is just data after all - and cohesive, well normalized data structures are desirable.
You'll notice that I don't include the tests in the installed packages. Because:
  • Module discovery tools will trip over your test modules. Strange things usually happen in test module. The help builtin does module discovery. E.g.:
    >>> help('modules')
    Please wait a moment while I gather a list of all available modules...
    
    __future__          antigravity         html                select
    ...
    
  • Tests usually require additional dependencies to run, so they aren't useful by their own - you can't run them directly.
  • Tests are concerned with development, not usage.
  • It's extremely unlikely that the user of the library will run the tests instead of the library's developer. E.g.: you don't run the tests for Django while testing your apps - Django is already tested.

Alternatives

You could use src-less layouts, few examples:
Tests in package Tests outside package
├─ packagename
│  ├─ __init__.py
│  ├─ ...
│  └─ tests
│     └─ ...
└─ setup.py
├─ packagename
│  ├─ __init__.py
│  └─ ...
├─ tests
│  └─ ...
└─ setup.py
These two layouts became popular because packaging had many problems few years ago, so it wasn't feasible to install the package just to test it. People still recommend them [4] even if it based on old and oudated assumptions.
Most projects use them incorectly, as all the test runners except Twisted's trial have incorrect defaults for the current working directory - you're going to test the wrong code if you don't test the installed code. trial does the right thing by changing the working directory to something temporary, but most projects don't use trial.

The setup script

Unfortunately with the current packaging tools, there are many pitfalls. The setup.py script should be as simple as possible:
#!/usr/bin/env python
# -*- encoding: utf-8 -*-
from __future__ import absolute_import
from __future__ import print_function

import io
import re
from glob import glob
from os.path import basename
from os.path import dirname
from os.path import join
from os.path import splitext

from setuptools import find_packages
from setuptools import setup


def read(*names, **kwargs):
    return io.open(
        join(dirname(__file__), *names),
        encoding=kwargs.get('encoding', 'utf8')
    ).read()


setup(
    name='nameless',
    version='0.1.0',
    license='BSD 2-Clause License',
    description='An example package. Generated with https://github.com/ionelmc/cookiecutter-pylibrary',
    long_description='%s\n%s' % (
        re.compile('^.. start-badges.*^.. end-badges', re.M | re.S).sub('', read('README.rst')),
        re.sub(':[a-z]+:`~?(.*?)`', r'``\1``', read('CHANGELOG.rst'))
    ),
    author='Ionel Cristian Mărieș',
    author_email='contact@ionelmc.ro',
    url='https://github.com/ionelmc/python-nameless',
    packages=find_packages('src'),
    package_dir={'': 'src'},
    py_modules=[splitext(basename(path))[0] for path in glob('src/*.py')],
    include_package_data=True,
    zip_safe=False,
    classifiers=[
        # complete classifier list: http://pypi.python.org/pypi?%3Aaction=list_classifiers
        'Development Status :: 5 - Production/Stable',
        'Intended Audience :: Developers',
        'License :: OSI Approved :: Apache Software License',
        'Operating System :: Unix',
        'Operating System :: POSIX',
        'Operating System :: Microsoft :: Windows',
        'Programming Language :: Python',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.3',
        'Programming Language :: Python :: 3.4',
        'Programming Language :: Python :: 3.5',
        'Programming Language :: Python :: 3.6',
        'Programming Language :: Python :: Implementation :: CPython',
        'Programming Language :: Python :: Implementation :: PyPy',
        # uncomment if you test on these interpreters:
        # 'Programming Language :: Python :: Implementation :: IronPython',
        # 'Programming Language :: Python :: Implementation :: Jython',
        # 'Programming Language :: Python :: Implementation :: Stackless',
        'Topic :: Utilities',
    ],
    keywords=[
        # eg: 'keyword1', 'keyword2', 'keyword3',
    ],
    install_requires=[
        'click',
        # eg: 'aspectlib==1.1.1', 'six>=1.7',
    ],
    extras_require={
        # eg:
        #   'rst': ['docutils>=0.11'],
        #   ':python_version=="2.6"': ['argparse'],
    },
    entry_points={
        'console_scripts': [
            'nameless = nameless.cli:main',
        ]
    },
)
What's special about this:
  • No exec or import trickery.
  • Includes everything from src: packages or root-level modules.
  • Explicit encodings.

Running the tests

Again, it seems people fancy the idea of running python setup.py test to run the package's tests. I think that's not worth doing - setup.py test is a failed experiment to replicate some of CPAN's test system. Python doesn't have a common test result protocol so it serves no purpose to have a common test command [1]. At least not for now - we'd need someone to build specifications and services that make this worthwhile, and champion them. I think it's important in general to recognize failure where there is and go back to the drawing board when that's necessary - there are absolutely no services or tools that use setup.py test command in a way that brings added value. Something is definitely wrong here.
I believe it's too late now for PyPI to do anything about it, Travis is already a solid, reliable, extremely flexible and free alternative. It integrates very well with Github - builds will be run automatically for each Pull Request.
To test locally tox is a very good way to run all the possible testing configurations (each configuration will be a tox environment). I like to organize the tests into a matrix with these additional environments:
  • check - check package metadata (e.g.: if the restructured text in your long description is valid)
  • clean - clean coverage
  • report - make coverage report for all the accumulated data
  • docs - build sphinx docs
I also like to have environments with and without coverage measurement and run them all the time. Race conditions are usually performance sensitive and you're unlikely to catch them if you run everything with coverage measurements.

The test matrix

Depending on dependencies you'll usually end up with a huge number of combinations of python versions, dependency versions and different settings. Generally people just hard-code everything in tox.ini or only in .travis.yml. They end up with incomplete local tests, or test configurations that run serially in Travis. I've tried that, didn't like it. I've tried duplicating the environments in both tox.ini and .travis.yml. Still didn't like it.
Note
This bootstrap.py technique is a bit outdated now. It still works fine but for simple matrices you can use a tox generative envlist (it was implemented after I wrote this blog post, unfortunately).

See python-nameless for an example using that.
As there were no readily usable alternatives to generate the configuration, I've implemented a generator script that uses templates to generate tox.ini and .travis.yml. This is way better, it's DRY, you can easily skip running tests on specific configurations (e.g.: skip Django 1.4 on Python 3) and there's less work to change things.
The essentials (full code):

setup.cfg

The generator script uses a configuration file (setup.cfg for convenience):
not_skip = __init__.py
skip = migrations

[matrix]
# This is the configuration for the `./bootstrap.py` script.
# It generates `.travis.yml`, `tox.ini` and `appveyor.yml`.
#
# Syntax: [alias:] value [!variable[glob]] [&variable[glob]]
#
# alias:
#  - is used to generate the tox environment
#  - it's optional
#  - if not present the alias will be computed from the `value`
# value:
#  - a value of "-" means empty
# !variable[glob]:
#  - exclude the combination of the current `value` with
#    any value matching the `glob` in `variable`
#  - can use as many you want
# &variable[glob]:
#  - only include the combination of the current `value`
#    when there's a value matching `glob` in `variable`
#  - can use as many you want

python_versions =
    2.7
    3.3
    3.4
    3.5
    3.6
    pypy

dependencies =
#    1.4: Django==1.4.16 !python_versions[3.*]
#    1.5: Django==1.5.11
#    1.6: Django==1.6.8
#    1.7: Django==1.7.1 !python_versions[2.6]
# Deps commented above are provided as examples. That's what you would use in a Django project.

coverage_flags =
    cover: true
    nocov: false

environment_variables =
    -

ci/bootstrap.py

This is the generator script. You run this whenever you want to regenerate the configuration:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import absolute_import, print_function, unicode_literals

import os
import sys
from os.path import abspath
from os.path import dirname
from os.path import exists
from os.path import join


if __name__ == "__main__":
    base_path = dirname(dirname(abspath(__file__)))
    print("Project path: {0}".format(base_path))
    env_path = join(base_path, ".tox", "bootstrap")
    if sys.platform == "win32":
        bin_path = join(env_path, "Scripts")
    else:
        bin_path = join(env_path, "bin")
    if not exists(env_path):
        import subprocess

        print("Making bootstrap env in: {0} ...".format(env_path))
        try:
            subprocess.check_call(["virtualenv", env_path])
        except subprocess.CalledProcessError:
            subprocess.check_call([sys.executable, "-m", "virtualenv", env_path])
        print("Installing `jinja2` and `matrix` into bootstrap environment...")
        subprocess.check_call([join(bin_path, "pip"), "install", "jinja2", "matrix"])
    activate = join(bin_path, "activate_this.py")
    # noinspection PyCompatibility
    exec(compile(open(activate, "rb").read(), activate, "exec"), dict(__file__=activate))

    import jinja2

    import matrix

    jinja = jinja2.Environment(
        loader=jinja2.FileSystemLoader(join(base_path, "ci", "templates")),
        trim_blocks=True,
        lstrip_blocks=True,
        keep_trailing_newline=True
    )

    tox_environments = {}
    for (alias, conf) in matrix.from_file(join(base_path, "setup.cfg")).items():
        python = conf["python_versions"]
        deps = conf["dependencies"]
        tox_environments[alias] = {
            "python": "python" + python if "py" not in python else python,
            "deps": deps.split(),
        }
        if "coverage_flags" in conf:
            cover = {"false": False, "true": True}[conf["coverage_flags"].lower()]
            tox_environments[alias].update(cover=cover)
        if "environment_variables" in conf:
            env_vars = conf["environment_variables"]
            tox_environments[alias].update(env_vars=env_vars.split())

    for name in os.listdir(join("ci", "templates")):
        with open(join(base_path, name), "w") as fh:
            fh.write(jinja.get_template(name).render(tox_environments=tox_environments))
        print("Wrote {}".format(name))
    print("DONE.")

ci/templates/.travis.yml

This has some goodies in it: the very useful libSegFault.so trick.
It basically just runs tox.
language: python
sudo: false
cache: pip
env:
  global:
    - LD_PRELOAD=/lib/x86_64-linux-gnu/libSegFault.so
    - SEGFAULT_SIGNALS=all
  matrix:
    - TOXENV=check
    - TOXENV=docs
matrix:
  include:
{%- for env, config in tox_environments|dictsort %}{{ '' }}
    - python: '{{ '{0[0]}-5.4'.format(env.split('-')) if env.startswith('pypy') else env.split('-')[0] }}'
      env:
        - TOXENV={{ env }}{% if config.cover %},report,coveralls,codecov{% endif -%}
{% endfor %}

before_install:
  - python --version
  - uname -a
  - lsb_release -a
install:
  - pip install tox
  - virtualenv --version
  - easy_install --version
  - pip --version
  - tox --version
script:
  - tox -v
after_failure:
  - more .tox/log/* | cat
  - more .tox/*/log/* | cat
notifications:
  email:
    on_success: never
    on_failure: always

ci/templates/tox.ini

[tox]
envlist =
    clean,
    check,
{% for env in tox_environments|sort %}
    {{ env }},
{% endfor %}
    report,
    docs

[testenv]
basepython =
    {docs,spell}: {env:TOXPYTHON:python2.7}
    {bootstrap,clean,check,report,extension-coveralls,coveralls,codecov}: {env:TOXPYTHON:python3}
setenv =
    PYTHONPATH={toxinidir}/tests
    PYTHONUNBUFFERED=yes
passenv =
    *
deps =
    pytest
    pytest-travis-fold
commands =
    {posargs:py.test -vv --ignore=src}

[testenv:spell]
setenv =
    SPELLCHECK=1
commands =
    sphinx-build -b spelling docs dist/docs
skip_install = true
usedevelop = false
deps =
    -r{toxinidir}/docs/requirements.txt
    sphinxcontrib-spelling
    pyenchant

[testenv:docs]
deps =
    -r{toxinidir}/docs/requirements.txt
commands =
    sphinx-build {posargs:-E} -b html docs dist/docs
    sphinx-build -b linkcheck docs dist/docs

[testenv:bootstrap]
deps =
    jinja2
    matrix
skip_install = true
usedevelop = false
commands =
    python ci/bootstrap.py
[testenv:check]
deps =
    docutils
    check-manifest
    flake8
    readme-renderer
    pygments
    isort
skip_install = true
usedevelop = false
commands =
    python setup.py check --strict --metadata --restructuredtext
    check-manifest {toxinidir}
    flake8 src tests setup.py
    isort --verbose --check-only --diff --recursive src tests setup.py

[testenv:coveralls]
deps =
    coveralls
skip_install = true
usedevelop = false
commands =
    coveralls []

[testenv:codecov]
deps =
    codecov
skip_install = true
usedevelop = false
commands =
    coverage xml --ignore-errors
    codecov []


[testenv:report]
deps = coverage
skip_install = true
usedevelop = false
commands =
    coverage combine --append
    coverage report
    coverage html

[testenv:clean]
commands = coverage erase
skip_install = true
usedevelop = false
deps = coverage

{% for env, config in tox_environments|dictsort %}
[testenv:{{ env }}]
basepython = {env:TOXPYTHON:{{ config.python }}}
{% if config.cover or config.env_vars %}
setenv =
    {[testenv]setenv}
{% endif %}
{% for var in config.env_vars %}
    {{ var }}
{% endfor %}
{% if config.cover %}
usedevelop = true
commands =
    {posargs:py.test --cov --cov-report=term-missing -vv}
{% endif %}
{% if config.cover or config.deps %}
deps =
    {[testenv]deps}
{% endif %}
{% if config.cover %}
    pytest-cov
{% endif %}
{% for dep in config.deps %}
    {{ dep }}
{% endfor %}

{% endfor %}

ci/templates/appveyor.ini

For Windows-friendly projects:
version: '{branch}-{build}'
build: off
cache:
  - '%LOCALAPPDATA%\pip\Cache'
environment:
  global:
    WITH_COMPILER: 'cmd /E:ON /V:ON /C .\ci\appveyor-with-compiler.cmd'
  matrix:
    - TOXENV: check
      TOXPYTHON: C:\Python27\python.exe
      PYTHON_HOME: C:\Python27
      PYTHON_VERSION: '2.7'
      PYTHON_ARCH: '32'
{% for env, config in tox_environments|dictsort %}{{ '' }}{% if config.python.startswith('python') %}
    - TOXENV: '{{ env }}{% if config.cover %},report,codecov{% endif %}'
      TOXPYTHON: C:\{{ config.python.replace('.', '').capitalize() }}\python.exe
      PYTHON_HOME: C:\{{ config.python.replace('.', '').capitalize() }}
      PYTHON_VERSION: '{{ config.python[-3:] }}'
      PYTHON_ARCH: '32'
    - TOXENV: '{{ env }}{% if config.cover %},report,codecov{% endif %}'
      TOXPYTHON: C:\{{ config.python.replace('.', '').capitalize() }}-x64\python.exe
      {%- if config.python != 'python3.5' %}

      WINDOWS_SDK_VERSION: v7.{{ '1' if config.python[-3] == '3' else '0' }}
      {%- endif %}

      PYTHON_HOME: C:\{{ config.python.replace('.', '').capitalize() }}-x64
      PYTHON_VERSION: '{{ config.python[-3:] }}'
      PYTHON_ARCH: '64'

{% endif %}{% endfor %}
init:
  - ps: echo $env:TOXENV
  - ps: ls C:\Python*
install:
  - python -u ci\appveyor-bootstrap.py
  - '%PYTHON_HOME%\Scripts\virtualenv --version'
  - '%PYTHON_HOME%\Scripts\easy_install --version'
  - '%PYTHON_HOME%\Scripts\pip --version'
  - '%PYTHON_HOME%\Scripts\tox --version'
test_script:
  - '%WITH_COMPILER% %PYTHON_HOME%\Scripts\tox'

on_failure:
  - ps: dir "env:"
  - ps: get-content .tox\*\log\*
artifacts:
  - path: dist\*

### To enable remote debugging uncomment this (also, see: http://www.appveyor.com/docs/how-to/rdp-to-build-worker):
# on_finish:
#   - ps: $blockRdp = $true; iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))
If you've been patient enough to read through that you'll notice:
  • The Travis configuration uses tox for each item in the matrix. This makes testing in Travis consistent with testing locally.
  • The environment order for tox is clean, check, 2.6-1.3, 2.6-1.4, ..., report.
  • The environments with coverage measurement run the code without installing (usedevelop = true) so that coverage can combine all the measurements at the end.
  • The environments without coverage will sdist and install into virtualenv (tox's default behavior [2]) so that packaging issues are caught early.
  • The report environment combines all the runs at the end into a single report.
Having the complete list of environments in tox.ini is a huge advantage:
  • You run everything in parallel locally (if your tests don't need strict isolation) with detox. And you can still run everything in parallel if you want to use drone.io instead of Travis.
  • You can measure cummulated coverage for everything (merge the coverage measurements for all the environments into a single one) locally.

Test coverage

There's Coveralls - a nice way to track coverage over time and over multiple builds. It will automatically add comments on Github Pull Request about changes in coverage.

TL;DR

  • Put code in src.
  • Use tox and detox.
  • Test both with coverage measurements and without.
  • Use a generator script for tox.ini and .travis.ini.
  • Run the tests in Travis with tox to keep things consistent with local testing.
Too complicated? Just use a python package template.
Not convincing enough? Read Hynek's post about the src layout.

Investigate Avro usage instead of json

https://avro.apache.org/docs/1.8.1/gettingstartedpython.html

FlatPak dev

https://blogs.gnome.org/alexl/

Cognitive bias cheat sheet

source

Cognitive bias cheat sheet

Because thinking is hard.

http://chainsawsuit.com/comic/2014/09/16/on-research/
I’ve spent many years referencing Wikipedia’s list of cognitive biases whenever I have a hunch that a certain type of thinking is an official bias but I can’t recall the name or details. It’s been an invaluable reference for helping me identify the hidden flaws in my own thinking. Nothing else I’ve come across seems to be both as comprehensive and as succinct.
However, honestly, the Wikipedia page is a bit of a tangled mess. Despite trying to absorb the information of this page many times over the years, very little of it seems to stick. I often scan it and feel like I’m not able to find the bias I’m looking for, and then quickly forget what I’ve learned. I think this has to do with how the page has organically evolved over the years. Today, it groups 175 biases into vague categories (decision-making biases, social biases, memory errors, etc) that don’t really feel mutually exclusive to me, and then lists them alphabetically within categories. There are duplicates a-plenty, and many similar biases with different names, scattered willy-nilly.
I’ve taken some time over the last four weeks (I’m on paternity leave) to try to more deeply absorb and understand this list, and to try to come up with a simpler, clearer organizing structure to hang these biases off of. Reading deeply about various biases has given my brain something to chew on while I bounce little Louie to sleep.
I started with the raw list of the 175 biases and added them all to a spreadsheet, then took another pass removing duplicates, and grouping similar biases (like bizarreness effect and humor effect) or complementary biases (like optimism bias and pessimism bias). The list came down to about 20 unique biased mental strategies that we use for very specific reasons.
I made several different attempts to try to group these 20 or so at a higher level, and eventually landed on grouping them by the general mental problem that they were attempting to address. Every cognitive bias is there for a reason — primarily to save our brains time or energy. If you look at them by the problem they’re trying to solve, it becomes a lot easier to understand why they exist, how they’re useful, and the trade-offs (and resulting mental errors) that they introduce.

Four problems that biases help us address:

Information overload, lack of meaning, the need to act fast, and how to know what needs to be remembered for later.

Problem 1: Too much information.

There is just too much information in the world, we have no choice but to filter almost all of it out. Our brain uses a few simple tricks to pick out the bits of information that are most likely going to be useful in some way.

Problem 2: Not enough meaning.

The world is very confusing, and we end up only seeing a tiny sliver of it, but we need to make some sense of it in order to survive. Once the reduced stream of information comes in, we connect the dots, fill in the gaps with stuff we already think we know, and update our mental models of the world.

Problem 3: Need to act fast.

We’re constrained by time and information, and yet we can’t let that paralyze us. Without the ability to act fast in the face of uncertainty, we surely would have perished as a species long ago. With every piece of new information, we need to do our best to assess our ability to affect the situation, apply it to decisions, simulate the future to predict what might happen next, and otherwise act on our new insight.

Problem 4: What should we remember?

There’s too much information in the universe. We can only afford to keep around the bits that are most likely to prove useful in the future. We need to make constant bets and trade-offs around what we try to remember and what we forget. For example, we prefer generalizations over specifics because they take up less space. When there are lots of irreducible details, we pick out a few standout items to save and discard the rest. What we save here is what is most likely to inform our filters related to problem 1’s information overload, as well as inform what comes to mind during the processes mentioned in problem 2 around filling in incomplete information. It’s all self-reinforcing.

Great, how am I supposed to remember all of this?

You don’t have to. But you can start by remembering these four giant problems our brains have evolved to deal with over the last few million years (and maybe bookmark this page if you want to occasionally reference it for the exact bias you’re looking for):
  1. Information overload sucks, so we aggressively filter. Noise becomes signal.
  2. Lack of meaning is confusing, so we fill in the gaps. Signal becomes a story.
  3. Need to act fast lest we lose our chance, so we jump to conclusions. Stories become decisions.
  4. This isn’t getting easier, so we try to remember the important bits. Decisions inform our mental models of the world.
In order to avoid drowning in information overload, our brains need to skim and filter insane amounts of information and quickly, almost effortlessly, decide which few things in that firehose are actually important and call those out.
In order to construct meaning out of the bits and pieces of information that come to our attention, we need to fill in the gaps, and map it all to our existing mental models. In the meantime we also need to make sure that it all stays relatively stable and as accurate as possible.
In order to act fast, our brains need to make split-second decisions that could impact our chances for survival, security, or success, and feel confident that we can make things happen.
And in order to keep doing all of this as efficiently as possible, our brains need to remember the most important and useful bits of new information and inform the other systems so they can adapt and improve over time, but no more than that.

Sounds pretty useful! So what’s the downside?

In addition to the four problems, it would be useful to remember these four truths about how our solutions to these problems have problems of their own:
  1. We don’t see everything. Some of the information we filter out is actually useful and important.
  2. Our search for meaning can conjure illusions. We sometimes imagine details that were filled in by our assumptions, and construct meaning and stories that aren’t really there.
  3. Quick decisions can be seriously flawed. Some of the quick reactions and decisions we jump to are unfair, self-serving, and counter-productive.
  4. Our memory reinforces errors. Some of the stuff we remember for later just makes all of the above systems more biased, and more damaging to our thought processes.
By keeping the four problems with the world and the four consequences of our brain’s strategy to solve them, the availability heuristic (and, specifically, the Baader-Meinhof phenomenon) will insure that we notice our own biases more often. If you visit this page to refresh your mind every once in a while, the spacing effect will help underline some of these thought patterns so that our bias blind spot and naïve realism is kept in check.
Nothing we do can make the 4 problems go away (until we have a way to expand our minds’ computational power and memory storage to match that of the universe) but if we accept that we are permanently biased, but that there’s room for improvement, confirmation bias will continue to help us find evidence that supports this, which will ultimately lead us to better understanding ourselves.
"Since learning about confirmation bias, I keep seeing it everywhere!”
Cognitive biases are just tools, useful in the right contexts, harmful in others. They’re the only tools we’ve got, and they’re even pretty good at what they’re meant to do. We might as well get familiar with them and even appreciate that we at least have some ability to process the universe with our mysterious brains.
Update: A couple days after posting this, John Manoogian III asked if it would be okay to do a “diagrammatic poster remix” of it, to which I of course said YES to. Here’s what he came up with:
If you feel so inclined, you can buy a poster-version of the above image here. If you want to play around with the data in JSON format, you can do that here.

🚀 Get more news about biases!

To get notifications about future posts and cognitive bias-related news, sign up here. And if you’d like to participate in my call for participants to be included in the book I’m writing about biases, become my patron here (it’s only $1/month).
I’ll leave you with the first part of this little poem by Emily Dickinson:
The Brain — is wider — than the Sky
For — put them side by side — 
The one the other will contain
With ease — and You — beside —