Paper: Consistency and matching without replacement

Paper: Assessing Treatment Effect Variation in Observational Studies: Results from a Data Challenge

Paper: Genome-wide Causation Studies of Complex Diseases

Article: Introducing the do-sampler for causal inference

Paper: Feature Selection via Mutual Information: New Theoretical Insights

Paper: A discriminative approach for finding and characterizing positivity violations using decision trees

Continue Reading…

Collapse

Read More

How did the role of Chief Data Officer come to drive data literacy at companies around the world? Find out how it all began in this interview with the first who held the title at Yahoo!

Continue Reading…

Collapse

Read More

Game Source, Procurement Graph, Data Moats, and Antitrust Regulation

1. Game Source Code -- Internet Archive has a collection of video game source code. The majority of these titles were originally released as commercial products and the source code was made available to the public at a later time.
2. European Public Procurement Knowledge Graph -- over 23 million triples (records), covering information about almost 220,000 tenders, built to support competitiveness and accountability by TheyBuyForYou. (via University of Southampton)
3. The Empty Promise of Data Moats (Andreessen-Horowitz) -- business model wonks reckoned that "data network effects" were a thing, but the benefits seen by companies claiming data network effects seem to be the benefits of simply having a lot of data. And that's not as defensible as hoped. I liked this essay.
4. Why Big Tech Keeps Outsmarting Antitrust Regulators (Tim O'Reilly) -- designers of marketplace-platform algorithms and screen layouts can arbitrarily allocate value to whom they choose. The marketplace is designed and controlled by its owners, and that design shapes “who gets what and why.” [...] Power over sellers ultimately translates into power over customers as well. When it comes to antitrust, the question of market power must be answered by analyzing the effect of these marketplace designs on both buyers and sellers, and how they change over time. How much of the value goes to the platform, how much to consumers, and how much to suppliers?

Continue reading Four short links: 22 July 2019.

Continue Reading…

Collapse

Read More

Every large organization is investing heavily in building data solutions and tools. They are building data solutions from scratch when they could be taking advantage of readily available tools and solutions. Many organizations are re-inventing the wheel and wasting resources.

Continue Reading…

Collapse

Read More

In this tutorial, you will learn about learning rate schedules and decay using Keras. You’ll learn how to use Keras’ standard learning rate decay along with step-based, linear, and polynomial learning rate schedules.

When training a neural network, the learning rate is often the most important hyperparameter for you to tune:

• Too small a learning rate and your neural network may not learn at all
• Too large a learning rate and you may overshoot areas of low loss (or even overfit from the start of training)

When it comes to training a neural network, the most bang for your buck (in terms of accuracy) is going to come from selecting the correct learning rate and appropriate learning rate schedule.

But that’s easier said than done.

To help deep learning practitioners such as yourself learn how to assess a problem and choose an appropriate learning rate, we’ll be starting a series of tutorials on learning rate schedules, decay, and hyperparameter tuning with Keras.

By the end of this series, you’ll have a good understanding of how to appropriately and effectively apply learning rate schedules with Keras to your own deep learning projects.

To learn how to use Keras for learning rate schedules and decay, just keep reading

Keras learning rate schedules and decay

In the first part of this guide, we’ll discuss why the learning rate is the most important hyperparameter when it comes to training your own deep neural networks.

We’ll then dive into why we may want to adjust our learning rate during training.

From there I’ll show you how to implement and utilize a number of learning rate schedules with Keras, including:

• The decay schedule built into most Keras optimizers
• Step-based learning rate schedules
• Linear learning rate decay
• Polynomial learning rate schedules

We’ll then perform a number of experiments on the CIFAR-10 using these learning rate schedules and evaluate which one performed the best.

These sets of experiments will serve as a template you can use when exploring your own deep learning projects and selecting an appropriate learning rate and learning rate schedule.

Why adjust our learning rate and use learning rate schedules?

To see why learning rate schedules are a worthwhile method to apply to help increase model accuracy and descend into areas of lower loss, consider the standard weight update formula used by nearly all neural networks:

Recall that the learning rate, , controls the “step” we make along the gradient. Larger values of imply that we are taking bigger steps. While smaller values of will make tiny steps. If is zero the network cannot make any steps at all (since the gradient multiplied by zero is zero).

Most initial learning rates (but not all) you encounter are typically in the set .

A network is then trained for a fixed number of epochs without changing the learning rate.

This method may work well in some situations, but it’s often beneficial to decrease our learning rate over time. When training our network, we are trying to find some location along our loss landscape where the network obtains reasonable accuracy. It doesn’t have to be a global minima or even a local minima, but in practice, simply finding an area of the loss landscape with reasonably low loss is “good enough”.

If we constantly keep a learning rate high, we could overshoot these areas of low loss as we’ll be taking too large of steps to descend into those series.

Instead, what we can do is decrease our learning rate, thereby allowing our network to take smaller steps — this decreased learning rate enables our network to descend into areas of the loss landscape that are “more optimal” and would have otherwise been missed entirely by our learning rate learning.

We can, therefore, view the process of learning rate scheduling as:

1. Finding a set of reasonably “good” weights early in the training process with a larger learning rate.
2. Tuning these weights later in the process to find more optimal weights using a smaller learning rate.

We’ll be covering some of the most popular learning rate schedules in this tutorial.

Project structure

Once you’ve grabbed and extracted the “Downloads” go ahead and use the

tree

$ tree
.
├── output
│   ├── lr_linear_schedule.png
│   ├── lr_poly_schedule.png
│   ├── lr_step_schedule.png
│   ├── train_linear_schedule.png
│   ├── train_no_schedule.png
│   ├── train_poly_schedule.png
│   ├── train_standard_schedule.png
│   └── train_step_schedule.png
├── pyimagesearch
│   ├── __init__.py
│   ├── learning_rate_schedulers.py
│   └── resnet.py
└── train.py

2 directories, 12 files

Our

output/

train_*.png

The

pyimagesearch

learning_rate_schedulers.py

LearningRateDecay

plot

StepDecay

PolynomialDecay

plot

Our training script,

train.py

The standard “decay” schedule in Keras

The Keras library ships with a time-based learning rate scheduler — it is controlled via the

decay

SGD

Adam

To discover how we can utilize this type of learning rate decay, let’s take a look at an example of how we may initialize the ResNet architecture and the SGD optimizer:

# initialize our optimizer and model, then compile it
opt = SGD(lr=1e-2, momentum=0.9, decay=1e-2/epochs)
model = ResNet.build(32, 32, 3, 10, (9, 9, 9),
	(64, 64, 128, 256), reg=0.0005)
model.compile(loss="categorical_crossentropy", optimizer=opt,
	metrics=["accuracy"])

Here we initialize our SGD optimizer with an initial learning rate of

1e-2

decay

Internally, Keras applies the following learning rate schedule to adjust the learning rate after every batch update — it is a misconception that Keras updates the standard decay after every epoch. Keep this in mind when using the default learning rate scheduler supplied with Keras.

The update formula follows:

Using the CIFAR-10 dataset as an example, we have a total of 50,000 training images.

If we use a batch size of

64

782

To see an example of the learning rate schedule calculation, let’s assume our initial learning rate is and our (with the assumption that we are training for forty epochs).

The learning rate at step zero, before any learning rate schedule has been applied, is:

At the beginning of epoch one we can see the following learning rate:

Figure 1 below continues the calculation of Keras’ standard learning rate decay and a decay of :

Figure 1: Keras’ standard learning rate decay table.

You’ll learn how to utilize this type of learning rate decay inside the “Implementing our training script” and “Keras learning rate schedule results” sections of this post, respectively.

Our LearningRateDecay class

In the remainder of this tutorial, we’ll be implementing our own custom learning rate schedules and then incorporating them with Keras when training our neural networks.

To keep our code neat and tidy, and not to mention, follow object-oriented programming best practices, let’s first define a base

LearningRateDecay

Open up the

learning_rate_schedulers.py

# import the necessary packages
import matplotlib.pyplot as plt
import numpy as np

class LearningRateDecay:
	def plot(self, epochs, title="Learning Rate Schedule"):
		# compute the set of learning rates for each corresponding
		# epoch
		lrs = [self(i) for i in epochs]

		# the learning rate schedule
		plt.style.use("ggplot")
		plt.figure()
		plt.plot(epochs, lrs)
		plt.title(title)
		plt.xlabel("Epoch #")
		plt.ylabel("Learning Rate")

Each and every learning rate schedule we implement will have a plot function, enabling us to visualize our learning rate over time.

With our base

LearningRateSchedule

Step-based learning rate schedules with Keras

Figure 2: Keras learning rate step-based decay. The schedule in red is a decay factor of 0.5 and blue is a factor of 0.25.

One popular learning rate scheduler is step-based decay where we systematically drop the learning rate after specific epochs during training.

The step decay learning rate scheduler can be seen as a piecewise function, as visualized in Figure 2 — here the learning rate is constant for a number of epochs, then drops, is constant once more, then drops again, etc.

When applying step decay to our learning rate, we have two options:

1. Define an equation that models the piecewise drop-in learning rate that we wish to achieve.
2. Use what I call the

   ctrl + c

   method to train a deep neural network. Here we train for some number of epochs at a given learning rate and eventually notice validation performance stagnating/stalling, then 

   ctrl + c

   to stop the script, adjust our learning rate, and continue training.

We’ll primarily be focusing on the equation-based piecewise drop to learning rate scheduling in this post.

The

ctrl + c

If you’d like to learn more about the

ctrl + c

When applying step decay, we often drop our learning rate by either (1) half or (2) an order of magnitude after every fixed number of epochs. For example, let’s suppose our initial learning rate is .

After 10 epochs we drop the learning rate to .

After another 10 epochs (i.e., the 20th total epoch), is dropped by a factor of

0.5

In fact, this is the exact same learning rate schedule that is depicted in Figure 2 (red line).

The blue line displays a more aggressive drop factor of

0.25

Where is the initial learning rate, is the factor value controlling the rate in which the learning date drops, D is the “Drop every” epochs value, and E is the current epoch.

The larger our factor is, the slower the learning rate will decay.

Conversely, the smaller the factor , the faster the learning rate will decay.

All that said, let’s go ahead and implement our

StepDecay

Go back to your

learning_rate_schedulers.py

class StepDecay(LearningRateDecay):
	def __init__(self, initAlpha=0.01, factor=0.25, dropEvery=10):
		# store the base initial learning rate, drop factor, and
		# epochs to drop every
		self.initAlpha = initAlpha
		self.factor = factor
		self.dropEvery = dropEvery

	def __call__(self, epoch):
		# compute the learning rate for the current epoch
		exp = np.floor((1 + epoch) / self.dropEvery)
		alpha = self.initAlpha * (self.factor ** exp)

		# return the learning rate
		return float(alpha)

Line 20 defines the constructor to our

StepDecay

initAlpha

dropEvery

The

__call__

• Accepts the current

  epoch

    number.
• Computes the learning rate based on the step-based decay formula detailed above (Lines 29 and 30).
• Returns the computed learning rate for the current epoch (Line 33).

You’ll see how to use this learning rate schedule later in this post.

Linear and polynomial learning rate schedules in Keras

Two of my favorite learning rate schedules are linear learning rate decay and polynomial learning rate decay.

Using these methods our learning rate is decayed to zero over a fixed number of epochs.

The rate in which the learning rate is decayed is based on the parameters to the polynomial function. A smaller exponent/power to the polynomial will cause the learning rate to decay “more slowly”, whereas larger exponents decay the learning rate “more quickly”.

Conveniently, both of these methods can be implemented in a single class:

class PolynomialDecay(LearningRateDecay):
	def __init__(self, maxEpochs=100, initAlpha=0.01, power=1.0):
		# store the maximum number of epochs, base learning rate,
		# and power of the polynomial
		self.maxEpochs = maxEpochs
		self.initAlpha = initAlpha
		self.power = power

	def __call__(self, epoch):
		# compute the new learning rate based on polynomial decay
		decay = (1 - (epoch / float(self.maxEpochs))) ** self.power
		alpha = self.initAlpha * decay

		# return the new learning rate
		return float(alpha)

Line 36 defines the constructor to our

PolynomialDecay

• maxEpochs

   : The total number of epochs we’ll be training for.

• initAlpha

   : The initial learning rate.

• power

   : The power/exponent of the polynomial.

Note that if you set

power=1.0

Lines 45 and 46 compute the adjusted learning rate for the current epoch while Line 49 returns the new learning rate.

Implementing our training script

Now that we’ve implemented a few different Keras learning rate schedules, let’s see how we can use them inside an actual training script.

Create a file named 

train.py

# set the matplotlib backend so figures can be saved in the background
import matplotlib
matplotlib.use("Agg")

# import the necessary packages
from pyimagesearch.learning_rate_schedulers import StepDecay
from pyimagesearch.learning_rate_schedulers import PolynomialDecay
from pyimagesearch.resnet import ResNet
from sklearn.preprocessing import LabelBinarizer
from sklearn.metrics import classification_report
from keras.callbacks import LearningRateScheduler
from keras.optimizers import SGD
from keras.datasets import cifar10
import matplotlib.pyplot as plt
import numpy as np
import argparse

Lines 2-16 import required packages. Line 3 sets the

matplotlib

• StepDecay

   : Our class which calculates and plots step-based learning rate decay.

• PolynomialDecay

   : The class we wrote to calculate polynomial-based learning rate decay.

• ResNet

   : Our Convolutional Neural Network implemented in Keras.

• LearningRateScheduler

   : A Keras callback. We’ll pass our learning rate

  schedule

    to this class which will be called as a callback at the completion of each epoch to calculate our learning rate.

Let’s move on and parse our command line arguments:

# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-s", "--schedule", type=str, default="",
	help="learning rate schedule method")
ap.add_argument("-e", "--epochs", type=int, default=100,
	help="# of epochs to train for")
ap.add_argument("-l", "--lr-plot", type=str, default="lr.png",
	help="path to output learning rate plot")
ap.add_argument("-t", "--train-plot", type=str, default="training.png",
	help="path to output training plot")
args = vars(ap.parse_args())

Our script accepts any of four command line arguments when the script is called via the terminal:

• --schedule

   : The learning rate schedule method. Valid options are “standard”, “step”, “linear”, “poly”. By default, no learning rate schedule will be used.

• --epochs

   : The number of epochs to train for (

  default=100

   ).

• --lr-plot

   : The path to the output plot. I suggest overriding the

  default

    of

  lr.png

    with a more descriptive path + filename.

• --train-plot

   : The path to the output accuracy/loss training history plot. Again, I suggest a descriptive path + filename, otherwise

  training.png

    will be set by

  default

   .

With our imports and command line arguments in hand, now it’s time to initialize our learning rate schedule:

# store the number of epochs to train for in a convenience variable,
# then initialize the list of callbacks and learning rate scheduler
# to be used
epochs = args["epochs"]
callbacks = []
schedule = None

# check to see if step-based learning rate decay should be used
if args["schedule"] == "step":
	print("[INFO] using 'step-based' learning rate decay...")
	schedule = StepDecay(initAlpha=1e-1, factor=0.25, dropEvery=15)

# check to see if linear learning rate decay should should be used
elif args["schedule"] == "linear":
	print("[INFO] using 'linear' learning rate decay...")
	schedule = PolynomialDecay(maxEpochs=epochs, initAlpha=1e-1, power=1)

# check to see if a polynomial learning rate decay should be used
elif args["schedule"] == "poly":
	print("[INFO] using 'polynomial' learning rate decay...")
	schedule = PolynomialDecay(maxEpochs=epochs, initAlpha=1e-1, power=5)

# if the learning rate schedule is not empty, add it to the list of
# callbacks
if schedule is not None:
	callbacks = [LearningRateScheduler(schedule)]

Line 33 sets the number of

epochs

args

callbacks

schedule

Lines 38-50 then select the learning rate

schedule

args["schedule"]

• "step"

   : Initializes

  StepDecay

   .

• "linear"

   : Initializes

  PolynomialDecay

    with

  power=1

    indicating that a linear learning rate decay will be utilized.

• "poly"

   : 

  PolynomialDecay

    with a

  power=5

    will be used.

After you’ve reproduced the results of the experiments in this tutorial, be sure to revisit Lines 38-50 and insert additional

elif

Lines 54 and 55 initialize the

LearningRateScheduler

callbacks

--schedule

Let’s go ahead and load our data:

# load the training and testing data, then scale it into the
# range [0, 1]
print("[INFO] loading CIFAR-10 data...")
((trainX, trainY), (testX, testY)) = cifar10.load_data()
trainX = trainX.astype("float") / 255.0
testX = testX.astype("float") / 255.0

# convert the labels from integers to vectors
lb = LabelBinarizer()
trainY = lb.fit_transform(trainY)
testY = lb.transform(testY)

# initialize the label names for the CIFAR-10 dataset
labelNames = ["airplane", "automobile", "bird", "cat", "deer",
	"dog", "frog", "horse", "ship", "truck"]

Line 60 loads our CIFAR-10 data. The dataset is conveniently already split into training and testing sets.

The only preprocessing we must perform is to scale the data into the range [0, 1] (Lines 61 and 62).

Lines 65-67 binarize the labels and then Lines 70 and 71 initialize our

labelNames

labelNames

Let’s initialize

decay

# initialize the decay for the optimizer
decay = 0.0

# if we are using Keras' "standard" decay, then we need to set the
# decay parameter
if args["schedule"] == "standard":
	print("[INFO] using 'keras standard' learning rate decay...")
	decay = 1e-1 / epochs

# otherwise, no learning rate schedule is being used
elif schedule is None:
	print("[INFO] no learning rate schedule being used")

Line 74 initializes our learning rate

decay

If we’re using the

"standard"

1e-1 / epochs

With all of our initializations taken care of, let’s go ahead and compile + train our

ResNet

# initialize our optimizer and model, then compile it
opt = SGD(lr=1e-1, momentum=0.9, decay=decay)
model = ResNet.build(32, 32, 3, 10, (9, 9, 9),
	(64, 64, 128, 256), reg=0.0005)
model.compile(loss="categorical_crossentropy", optimizer=opt,
	metrics=["accuracy"])

# train the network
H = model.fit(trainX, trainY, validation_data=(testX, testY),
	batch_size=128, epochs=epochs, callbacks=callbacks, verbose=1)

Our Stochastic Gradient Descent (

SGD

decay

From there, Lines 88 and 89 build our

ResNet

Our

model

loss

"categorical_crossentropy"

loss="binary_crossentropy"

Lines 94 and 95 kick of our training process. Notice that we’ve provided the

callbacks

callbacks

LearningRateScheduler

callbacks

Finally, let’s evaluate our network and generate plots:

# evaluate the network
print("[INFO] evaluating network...")
predictions = model.predict(testX, batch_size=128)
print(classification_report(testY.argmax(axis=1),
	predictions.argmax(axis=1), target_names=labelNames))

# plot the training loss and accuracy
N = np.arange(0, args["epochs"])
plt.style.use("ggplot")
plt.figure()
plt.plot(N, H.history["loss"], label="train_loss")
plt.plot(N, H.history["val_loss"], label="val_loss")
plt.plot(N, H.history["acc"], label="train_acc")
plt.plot(N, H.history["val_acc"], label="val_acc")
plt.title("Training Loss and Accuracy on CIFAR-10")
plt.xlabel("Epoch #")
plt.ylabel("Loss/Accuracy")
plt.legend()
plt.savefig(args["train_plot"])

# if the learning rate schedule is not empty, then save the learning
# rate plot
if schedule is not None:
	schedule.plot(N)
	plt.savefig(args["lr_plot"])

Lines 99-101 evaluate our network and print a classification report to our terminal.

Lines 104-115 generate and save our training history plot (accuracy/loss curves). Lines 119-121 generate a learning rate schedule plot, if applicable. We will inspect these plot visualizations in the next section.

Keras learning rate schedule results

With both our (1) learning rate schedules and (2) training scripts implemented, let’s run some experiments to see which learning rate schedule will perform best given:

1. An initial learning rate of

   1e-1

2. Training for a total of

   100

     epochs

Experiment #1: No learning rate decay/schedule

As a baseline, let’s first train our ResNet model on CIFAR-10 with no learning rate decay or schedule:

$ python train.py --train-plot output/train_no_schedule.png
[INFO] loading CIFAR-10 data...
[INFO] no learning rate being used
Train on 50000 samples, validate on 10000 samples
Epoch 1/100
50000/50000 [==============================] - 186s 4ms/step - loss: 2.1204 - acc: 0.4372 - val_loss: 1.9361 - val_acc: 0.5118
Epoch 2/100
50000/50000 [==============================] - 171s 3ms/step - loss: 1.5150 - acc: 0.6440 - val_loss: 1.5013 - val_acc: 0.6413
Epoch 3/100
50000/50000 [==============================] - 171s 3ms/step - loss: 1.2186 - acc: 0.7369 - val_loss: 1.2288 - val_acc: 0.7315
...
Epoch 98/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.5220 - acc: 0.9568 - val_loss: 1.0223 - val_acc: 0.8372
Epoch 99/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.5349 - acc: 0.9532 - val_loss: 1.0423 - val_acc: 0.8230
Epoch 100/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.5209 - acc: 0.9579 - val_loss: 0.9883 - val_acc: 0.8421
[INFO] evaluating network...
              precision    recall  f1-score   support

    airplane       0.84      0.86      0.85      1000
  automobile       0.90      0.93      0.92      1000
        bird       0.83      0.74      0.78      1000
         cat       0.67      0.79      0.73      1000
        deer       0.78      0.88      0.83      1000
         dog       0.85      0.69      0.76      1000
        frog       0.85      0.89      0.87      1000
       horse       0.94      0.82      0.88      1000
        ship       0.91      0.90      0.90      1000
       truck       0.90      0.90      0.90      1000

   micro avg       0.84      0.84      0.84     10000
   macro avg       0.85      0.84      0.84     10000
weighted avg       0.85      0.84      0.84     10000

Figure 3: Our first experiment for training ResNet on CIFAR-10 does not have learning rate decay.

Here we obtain ~85% accuracy, but as we can see, validation loss and accuracy stagnate past epoch ~15 and do not improve over the rest of the 100 epochs.

Our goal is now to utilize learning rate scheduling to beat our 85% accuracy (without overfitting).

Experiment: #2: Keras standard optimizer learning rate decay

In our second experiment we are going to use Keras’ standard decay-based learning rate schedule:

$ python train.py --schedule standard --train-plot output/train_standard_schedule.png
[INFO] loading CIFAR-10 data...
[INFO] using 'keras standard' learning rate decay...
Train on 50000 samples, validate on 10000 samples
Epoch 1/100
50000/50000 [==============================] - 184s 4ms/step - loss: 2.1074 - acc: 0.4460 - val_loss: 1.8397 - val_acc: 0.5334
Epoch 2/100
50000/50000 [==============================] - 171s 3ms/step - loss: 1.5068 - acc: 0.6516 - val_loss: 1.5099 - val_acc: 0.6663
Epoch 3/100
50000/50000 [==============================] - 171s 3ms/step - loss: 1.2097 - acc: 0.7512 - val_loss: 1.2928 - val_acc: 0.7176
...
Epoch 98/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.1752 - acc: 1.0000 - val_loss: 0.8892 - val_acc: 0.8209
Epoch 99/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.1746 - acc: 1.0000 - val_loss: 0.8923 - val_acc: 0.8204
Epoch 100/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.1740 - acc: 1.0000 - val_loss: 0.8924 - val_acc: 0.8208
[INFO] evaluating network...
              precision    recall  f1-score   support

    airplane       0.81      0.86      0.84      1000
  automobile       0.91      0.91      0.91      1000
        bird       0.75      0.71      0.73      1000
         cat       0.68      0.65      0.66      1000
        deer       0.78      0.81      0.79      1000
         dog       0.77      0.74      0.75      1000
        frog       0.83      0.88      0.85      1000
       horse       0.86      0.87      0.86      1000
        ship       0.90      0.90      0.90      1000
       truck       0.90      0.88      0.89      1000

   micro avg       0.82      0.82      0.82     10000
   macro avg       0.82      0.82      0.82     10000
weighted avg       0.82      0.82      0.82     10000

Figure 4: Our second learning rate decay schedule experiment uses Keras’ standard learning rate decay schedule.

This time we only obtain 82% accuracy, which goes to show, learning rate decay/scheduling will not always improve your results! You need to be careful which learning rate schedule you utilize.

Experiment #3: Step-based learning rate schedule results

Let’s go ahead and perform step-based learning rate scheduling which will drop our learning rate by a factor of 0.25 every 15 epochs:

$ python train.py --schedule step --lr-plot output/lr_step_schedule.png --train-plot output/train_step_schedule.png
[INFO] using 'step-based' learning rate decay...
[INFO] loading CIFAR-10 data...
Train on 50000 samples, validate on 10000 samples
Epoch 1/100
50000/50000 [==============================] - 186s 4ms/step - loss: 2.2839 - acc: 0.4328 - val_loss: 1.8936 - val_acc: 0.5530
Epoch 2/100
50000/50000 [==============================] - 171s 3ms/step - loss: 1.6425 - acc: 0.6213 - val_loss: 1.4599 - val_acc: 0.6749
Epoch 3/100
50000/50000 [==============================] - 171s 3ms/step - loss: 1.2971 - acc: 0.7177 - val_loss: 1.3298 - val_acc: 0.6953
...
Epoch 98/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.1817 - acc: 1.0000 - val_loss: 0.7221 - val_acc: 0.8653
Epoch 99/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.1817 - acc: 1.0000 - val_loss: 0.7228 - val_acc: 0.8661
Epoch 100/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.1817 - acc: 1.0000 - val_loss: 0.7267 - val_acc: 0.8652
[INFO] evaluating network...
              precision    recall  f1-score   support

    airplane       0.86      0.89      0.87      1000
  automobile       0.94      0.93      0.94      1000
        bird       0.83      0.80      0.81      1000
         cat       0.75      0.73      0.74      1000
        deer       0.82      0.87      0.84      1000
         dog       0.82      0.77      0.79      1000
        frog       0.89      0.90      0.90      1000
       horse       0.91      0.90      0.90      1000
        ship       0.93      0.93      0.93      1000
       truck       0.90      0.93      0.92      1000

   micro avg       0.87      0.87      0.87     10000
   macro avg       0.86      0.87      0.86     10000
weighted avg       0.86      0.87      0.86     10000

Figure 5: Experiment #3 demonstrates a step-based learning rate schedule (left). The training history accuracy/loss curves are shown on the right.

Figure 5 (left) visualizes our learning rate schedule. Notice how after every 15 epochs our learning rate drops, creating the “stair-step”-like effect.

Figure 5 (right) demonstrates the classic signs of step-based learning rate scheduling — you can clearly see our:

1. Training/validation loss decrease
2. Training/validation accuracy increase

…when our learning rate is dropped.

This is especially pronounced in the first two drops (epochs 15 and 30), after which the drops become less substantial.

This type of steep drop is a classic sign of a step-based learning rate schedule being utilized — if you see that type of training behavior in a paper, publication, or another tutorial, you can be almost sure that they used step-based decay!

Getting back to our accuracy, we’re now at 86-87% accuracy, an improvement from our first experiment.

Experiment #4: Linear learning rate schedule results

Let’s try using a linear learning rate schedule with Keras by setting 

power=1.0

$ python train.py --schedule linear --lr-plot output/lr_linear_schedule.png --train-plot output/train_linear_schedule.png
[INFO] using 'linear' learning rate decay...
[INFO] loading CIFAR-10 data...
Epoch 1/100
50000/50000 [==============================] - 187s 4ms/step - loss: 2.0399 - acc: 0.4541 - val_loss: 1.6900 - val_acc: 0.5789
Epoch 2/100
50000/50000 [==============================] - 171s 3ms/step - loss: 1.4623 - acc: 0.6588 - val_loss: 1.4535 - val_acc: 0.6557
Epoch 3/100
50000/50000 [==============================] - 171s 3ms/step - loss: 1.1790 - acc: 0.7480 - val_loss: 1.2633 - val_acc: 0.7230
...
Epoch 98/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.1025 - acc: 1.0000 - val_loss: 0.5623 - val_acc: 0.8804
Epoch 99/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.1021 - acc: 1.0000 - val_loss: 0.5636 - val_acc: 0.8800
Epoch 100/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.1019 - acc: 1.0000 - val_loss: 0.5622 - val_acc: 0.8808
[INFO] evaluating network...
              precision    recall  f1-score   support

    airplane       0.88      0.91      0.89      1000
  automobile       0.94      0.94      0.94      1000
        bird       0.84      0.81      0.82      1000
         cat       0.78      0.76      0.77      1000
        deer       0.86      0.90      0.88      1000
         dog       0.84      0.80      0.82      1000
        frog       0.90      0.92      0.91      1000
       horse       0.91      0.91      0.91      1000
        ship       0.93      0.94      0.93      1000
       truck       0.93      0.93      0.93      1000

   micro avg       0.88      0.88      0.88     10000
   macro avg       0.88      0.88      0.88     10000
weighted avg       0.88      0.88      0.88     10000

Figure 6: Linear learning rate decay (left) applied to ResNet on CIFAR-10 over 100 epochs with Keras. The training accuracy/loss curve is displayed on the right.

Figure 6 (left) shows that our learning rate is decreasing linearly over time while Figure 6 (right) visualizes our training history.

We’re now seeing a sharper drop in both training and validation loss, especially past approximately epoch 75; however, note that our training loss is dropping significantly faster than our validation loss — we may be at risk of overfitting.

Regardless, we are now obtaining 88% accuracy on our data, our best result thus far.

Experiment #5: Polynomial learning rate schedule results

As a final experiment let’s apply polynomial learning rate scheduling with Keras by setting

power=5

$ python train.py --schedule poly --lr-plot output/lr_poly_schedule.png --train-plot output/train_poly_schedule.png
[INFO] using 'polynomial' learning rate decay...
[INFO] loading CIFAR-10 data...
Epoch 1/100
50000/50000 [==============================] - 186s 4ms/step - loss: 2.0470 - acc: 0.4445 - val_loss: 1.7379 - val_acc: 0.5576
Epoch 2/100
50000/50000 [==============================] - 171s 3ms/step - loss: 1.4793 - acc: 0.6448 - val_loss: 1.4536 - val_acc: 0.6513
Epoch 3/100
50000/50000 [==============================] - 171s 3ms/step - loss: 1.2080 - acc: 0.7332 - val_loss: 1.2363 - val_acc: 0.7183
...
Epoch 98/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.1547 - acc: 1.0000 - val_loss: 0.6960 - val_acc: 0.8581
Epoch 99/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.1547 - acc: 1.0000 - val_loss: 0.6883 - val_acc: 0.8596
Epoch 100/100
50000/50000 [==============================] - 171s 3ms/step - loss: 0.1548 - acc: 1.0000 - val_loss: 0.6942 - val_acc: 0.8601
[INFO] evaluating network...
              precision    recall  f1-score   support

    airplane       0.86      0.89      0.87      1000
  automobile       0.94      0.94      0.94      1000
        bird       0.78      0.80      0.79      1000
         cat       0.75      0.70      0.73      1000
        deer       0.83      0.86      0.84      1000
         dog       0.81      0.78      0.79      1000
        frog       0.86      0.91      0.89      1000
       horse       0.92      0.88      0.90      1000
        ship       0.94      0.92      0.93      1000
       truck       0.91      0.92      0.91      1000

   micro avg       0.86      0.86      0.86     10000
   macro avg       0.86      0.86      0.86     10000
weighted avg       0.86      0.86      0.86     10000

Figure 7: Polynomial-based learning decay results using Keras.

Figure 7 (left) visualizes the fact that our learning rate is now decaying according to our polynomial function while Figure 7 (right) plots our training history.

This time we obtain ~86% accuracy.

Commentary on learning rate schedule experiments

Our best experiment was from our fourth experiment where we utilized a linear learning rate schedule.

But does that mean we should always use a linear learning rate schedule?

No, far from it, actually.

The key takeaway here is that for this:

• Particular dataset (CIFAR-10)
• Particular neural network architecture (ResNet)
• Initial learning rate of 1e-2
• Number of training epochs (100)

…is that linear learning rate scheduling worked the best.

No two deep learning projects are alike so you will need to run your own set of experiments, including varying the initial learning rate and the total number of epochs, to determine the appropriate learning rate schedule (additional commentary is included in the “Summary” section of this tutorial as well).

Do other learning rate schedules exist?

Other learning rate schedules exist, and in fact, any mathematical function that can accept an epoch or batch number as an input and returns a learning rate can be considered a “learning rate schedule”. Two other learning rate schedules you may encounter include (1) exponential learning rate decay, as well as (2) cyclical learning rates.

I don’t often use exponential decay as I find that linear and polynomial decay are more than sufficient, but you are more than welcome to subclass the

LearningRateDecay

Cyclical learning rates, on the other hand, are very powerful — we’ll be covering cyclical learning rates in a tutorial later in this series.

How do I choose my initial learning rate?

You’ll notice that in this tutorial we did not vary our learning rate, we kept it constant at

1e-2

When performing your own experiments you’ll want to combine:

1. Learning rate schedules…
2. …with different learning rates

Don’t be afraid to mix and match!

The four most important hyperparameters you’ll want to explore, include:

1. Initial learning rate
2. Number of training epochs
3. Learning rate schedule
4. Regularization strength/amount (L2, dropout, etc.)

Finding an appropriate balance of each can be challenging, but through many experiments, you’ll be able to find a recipe that leads to a highly accurate neural network.

If you’d like to learn more about my tips, suggestions, and best practices for learning rates, learning rate schedules, and training your own neural networks, refer to my book, Deep Learning for Computer Vision with Python.

Where can I learn more?

Figure 8: Deep Learning for Computer Vision with Python is a deep learning book for beginners, practitioners, and experts alike.

Today’s tutorial introduced you to learning rate decay and schedulers using Keras. To learn more about learning rates, schedulers, and how to write custom callback functions, refer to my book, Deep Learning for Computer Vision with Python.

Inside the book I cover:

1. More details on learning rates (and how a solid understanding of the concept impacts your deep learning success)
2. How to spot under/overfitting on-the-fly with a custom training monitor callback
3. How to checkpoint your models with a custom callback
4. My tips/tricks, suggestions, and best practices for training CNNs

Besides content on learning rates, you’ll also find:

• Super practical walkthroughs that present solutions to actual, real-world image classification, object detection, and instance segmentation problems.
• Hands-on tutorials (with lots of code) that not only show you the algorithms behind deep learning for computer vision but their implementations as well.
• A no-nonsense teaching style that is guaranteed to help you master deep learning for image understanding and visual recognition.

To learn more about the book, and grab the table of contents + free sample chapters, just click here!

Summary

In this tutorial, you learned how to utilize Keras for learning rate decay and learning rate scheduling.

Specifically, you discovered how to implement and utilize a number of learning rate schedules with Keras, including:

• The decay schedule built into most Keras optimizers
• Step-based learning rate schedules
• Linear learning rate decay
• Polynomial learning rate schedules

After implementing our learning rate schedules we evaluated each on a set of experiments on the CIFAR-10 dataset.

Our results demonstrated that for an initial learning rate of

1e-2

100

However, this does not mean that a linear learning rate schedule will always outperform other types of schedules. Instead, all this means is that for this:

• Particular dataset (CIFAR-10)
• Particular neural network architecture (ResNet)
• Initial learning rate of

  1e-2

• Number of training epochs (

  100

   )

…that linear learning rate scheduling worked the best.

No two deep learning projects are alike so you will need to run your own set of experiments, including varying the initial learning rate, to determine the appropriate learning rate schedule.

I suggest you keep an experiment log that details any hyperparameter choices and associated results, that way you can refer back to it and double-down on experiments that look promising.

Do not expect that you’ll be able to train a neural network and be “one and done” — that rarely, if ever, happens. Instead, set the expectation with yourself that you’ll be running many experiments and tuning hyperparameters as you go along. Machine learning, deep learning, and artificial intelligence as a whole are iterative — you build on your previous results.

Later in this series of tutorials I’ll also be showing you how to select your initial learning rate.

To download the source code to this post, and be notified when future tutorials are published here on PyImageSearch, just enter your email address in the form below!

Downloads:

If you would like to download the code and images used in this post, please enter your email address in the form below. Not only will you get a .zip of the code, I’ll also send you a FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning. Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL! Sound good? If so, enter your email address and I’ll send you the code immediately!

Email address:

Website

The post Keras learning rate schedules and decay appeared first on PyImageSearch.

Continue Reading…

Collapse

Read More

Simply stated, data sensemaking is what we do to make sense of data. We do this in an attempt to understand the world, based on empirical evidence. Those who work to make sense of data and communicate their findings are data sensemakers. Data sensemaking, as a profession, is currently associated with several job titles, including data analyst, business intelligence professional, statistician, and data scientist. Helping people understand the world based on data is important work. Without understanding, we often make bad decisions. When done well, data sensemaking requires a broad and deep set of skills and a commitment to ethical conduct. When data sensemaking professionals fail to do their jobs well, whether through a lack of skills or other ethical misconduct, confusion and misinformation results, which encourages bad decisions—decisions that do harm. Making sense of data is not ethically or morally neutral; it can be done for good or ill. “I did what I was told” is not a valid excuse for unethical behavior.

In recent years, misuses of data have led to a great deal of discussion about ethics related to invasions of privacy and discriminatory uses of data. Most of these discussions focus on the creation and use of analytical algorithms. I’d like to extend the list of ethical considerations to address the full range of data sensemaking activities. The list of ethical practices that I’m proposing below is neither complete nor sufficiently organized nor fully described. I offer it only as an initial effort that we can discuss, expand, and clarify. Once we’ve done that, we can circle back and refine the work.

The ethical practices that can serve as a code of conduct for data sensemaking professionals are, in my opinion, built upon a single fundamental principle. It is the same principle that medical doctors swear as an oath before becoming licensed: Do no harm.

Here’s the list:

1. You should work, not just to provide information, but to enable understanding that can be used in beneficial ways.
2. You should develop the full range of skills that are needed to do the work of data sensemaking effectively. Training in a data analysis tool is not sufficient. This suggests the need for an agreed-upon set of skills for data sensemaking.
3. You should understand the relevant domain. For instance, if you’re doing sales analysis, you should understand the sales process as well as the sales objectives of your organization. When you don’t understand the domain well enough, you must involve those who do.
4. You should know your audience (i.e., your clients; those who are asking you to do the work)—their interests, beliefs, values, assumptions, biases, and objectives—in part to identify potentially unethical inclinations.
5. You should understand the purpose for which your work will be used. In other words, you should ask “Why?”.
6. You should strive to anticipate the ways in which your findings could be used for harm.
7. When asked to do something harmful, you should say “No.” Furthermore, you should also discourage others from doing harm.
8. When you discover harmful uses of data, you should challenge them, and if they persist, you should expose them to those who can potentially end them.
9. You should primarily serve the needs of those who will be affected by your work, which is not necessarily those who have asked you to do the work.
10. You should not examine data that you or your client have no right to examine. This includes data that is private, which you have not received explicit permission to examine. To do this, you must acquaint yourself with data privacy laws, but not limit yourself to concern only for data that has been legally deemed private if it seems reasonable that it should be considered private nonetheless.
11. You should not do work that will result in the unfair and discriminatory treatment of particular groups of people based on race, ethnicity, gender, religion, age, etc.
12. If you cannot enable the understanding that’s needed with the data that’s available, you should point this out, identify what’s needed, and do what you can to acquire it.
13. If the quality of the data that’s available is insufficient for the data sensemaking task, you should point this out, describe what’s lacking, and insist that the data’s quality be improved to the level that’s required before proceeding.
14. You should always examine data within context.
15. You should always examine data from all potentially relevant perspectives.
16. You should present your findings clearly.
17. You should present your findings as comprehensively as necessary to enable the level of understanding that’s needed.
18. You should present your findings truthfully.
19. You should describe the uncertainty of your findings.
20. You should report any limitations that might have had an effect on the validity of your findings.
21. You should confirm that your audience understands your findings.
22. You should solicit feedback during the data sensemaking process and invite others to critique your findings.
23. You should document the steps that you took, including the statistics that you used, and maintain the data that you produced during the course of your work. This will make it possible for others to review your work and for you to reexamine your findings at a later date.
24. When you’re asked to do work that doesn’t make sense or to do it in a way that doesn’t make sense (i.e., in ways that are ineffective), you should propose an alternative that does make sense and insist on it.
25. When people telegraph what they expect you to find in the data, you should do your best to ignore those expectations or to subject them to scrutiny.
    As data sensemakers, we stand at the gates of understanding. Ethically, it is our job to serve as gatekeepers. In many cases, we will be the only defense against harm.

I invite you to propose additions to this list and to discuss the merits of the practices that I’ve proposed. If you are part of an organization that employs other data sensemakers, I also invite you to discuss the ethical dimensions of your work with one another.

Continue Reading…

Collapse

Read More

Alison Mattek writes:

I saw your recent blog post on falsifiable claims. For the past couple of years I have been developing a theoretical framework that highlights the importance of unfalsifiable claims in science. I try to also make a few unfalsifiable claims regarding psychological variables.

Here is Mattek’s paper, “Expanding psychological theory using system analogies.” It reminds me a bit of the writings of Paul Meehl.

Continue Reading…

Collapse

Read More

A sampling of radial chart submissions received this month

This month’s challenge was to find a dataset that makes sense to visualize in circular form—including but not limited to chord diagrams, coxcomb charts, polar area diagrams, radar plots, or sunbursts—and share what you learned about the process. We received over 50 examples of radial charts, with topics including exercise, climate change, activities performed during a 12 or 24-hour period, and music as the Tableau community recently finished creating 2019 Iron Viz qualifiers.

Participants were largely united in their evaluation of this challenge—finding a subject that fits a circular view is hard! Even harder was evaluating whether the final radial graph was effective at all. Many were quick to point out that they’d be inclined to choose a different type of graph, but appreciated the spirit of the exercise and realized they had learned something in the process. Our intent with the #SWDchallenge is to give the community a platform to practice trying something new in a low-risk environment. While this challenge seemed to push most of us outside of our comfort zones, that discomfort itself is immensely valuable for what it teaches us about our work and ourselves. Without experimentation, we never discover what unexpected approach might be applicable in an uncommon setting, or what technical skills we might enjoy developing. Conversely, it also gives us insight into what attempts are better suited for the intentional discard.

The key lesson from this month is that is that evaluating our own work can be a challenge—particularly when designing graphs for someone other than ourselves. In our workshops, we demonstrate one technique to step outside of your own head: solicit feedback from a friend or colleague. Show them your visual and have them talk you though their reactions. To which elements do they pay attention? What questions do they still have? This feedback process gives valuable input if your graph is working as you intended—and where to focus your rework if needed.

While radial charts may be aesthetically pleasing—humans are naturally drawn to circles and curves—circular visuals are often in conflict with the effective communication of data. If your goal is an attention-getting chart (without much concern over whether the data gets transformed into knowledge or action) then a radial chart may suffice. But keep in mind that you are more likely creating visual art than effective data visualization.

For those of you who did submit examples, THANK YOU for taking the time to create and share your work. The entries below are posted in alphabetical order by first name. If you tweeted or thought you submitted one but don't see it here, upload your submission as a .png here and we'll work to include any late entries this week (tweeting on its own isn't enough—we don't have time to scrape Twitter for entries.)

The next monthly challenge will launch on August 1st. Until then, check out the archives of previous challenges on our #SWDchallenge page. Happy practicing!

Adam

Showcasing a Radial Bar chart for this months SWDChallenge, deciding to present Michael Jackson's UK Peak charting singles positions, data sourced from Official Charts.com
Blog

Adina

I used the data from FiveThirtyEight's Halloween candy ranking story to create this sunburst diagram. The original story included a regression analysis of each candy attribute (chocolate, fruit, crispy, etc.), but the sunburst diagram allows the comparison of different attribute combinations, not just each one separately.

Allison

At first, I wasn't sure if I would attempt this month's challenge because I was nervous about creating a radial visualization. As someone who only opened Tableau for the first time in February 2019, is entirely self-taught, and hasn't thought about trigonometry since high school, I knew I had some work to do!

I began by considering what types of visualizations lend themselves well to this type of view. I tend to almost always err on the side of a simple bar chart to convey the big idea, but I decided to keep an open mind. I also wanted to tell a story about something that would be relatable and interesting. That’s when I thought about climate change. I realized that if I look at global warming over time, I could use a radial visualization to see interesting spikes and trends. I read through a bunch of tutorials and practiced creating radial charts (Kevin Flerlage’s tutorial was especially helpful). I’m not naturally the most mathematically inclined, so I spent way too long playing around with the radius, but I finally made it look right!

Past the radial viz, I wanted to provide a context of what this means for the world currently as well as in the future. I researched implications for continued warming of the world and shared outcomes that scientists think are likely if we reach a 2 degree Celsius warming. I also shared steps individuals can take to help combat climate change (however I really think governments and corporations need to make changes as well). Overall this was a fun challenge that helped me acquire lots of new skills!
LINK

Amanda

Radial charts proved to be a challenge for me, but I appreciate this blog pushing me to try something totally new. I am a sports fanatic, so when I found QB data online, I was excited to put it to use.

Anand

This is a network visualization connecting Directors and Companies in the Tata Group (public information). When the network stabilizes, it is interesting to see that it neatly segregates into two groups of companies. There are many common directors, but the network is clearly split. This was quite an interesting pattern to see.

Andy

Great fun. I've never built a radial chart before. I took 5 years of my music listening data and started with a matrix of Weekday/Year. In each cell, I drew a radial bar chart - one bar for each hour in the day. The longer the bar? The more tracks I listened to. I'm pleased with the outcome, but the static version is not readable. I could add some clever annotations to help. The interactive version does allow for insight. I added highlighting and a tutorial overlay to assist the interactive explorer. A bar chart version is way more readable, but even I have to admit the radial version is more appealing to the eye!
Blog

Angie

I was inspired by the Rhythm of Food example to look at how food interests change over time. Non-dairy milks are of special interest to me as my daughter has a milk allergy and will soon be at the age where she *would* be moving on to cows milk, but we will need to find an alternative, and I also am trying to better understand the environmental impact of the different non-dairy milks. Oat milk seems to frequently be out of stock at my local Whole Foods so it was no surprise to me that it was a very hot "milk" in terms of search, but I did not realize it was nearing the same level as soy milk. I did find the Dec to Jan spike across all milks quite interesting and unexpected. I used Excel to create the chart.

Arthur

Radial charts include many different types of of charts and I selected two of them for this challenge: the radial bar chart and nested pie chart. A radial bar chart, also known as a circular bar chart, displays the history of life on earth and provides context of the relative time periods for each form of live. The nested pie chart, rendered as a donut chart to display the earth image within, outlines the specific time periods and eras in which life came to be. Combined, they form an infographic where the sum of the parts not only convey the data, but also together represent the earth as it's own circular chart visual. The chart has been kept clean of labels and utilizes dynamic tooltips to show additional information the viewer is interested in on hover or tap. You can view the live radial chart to explore this interactivity directly.

Brian

Maximum duration of western hemisphere eclipses for years 2000-2099 grouped by month:

Catherine B

It was hard for me to find a subject with data that could be well represented with a circular chart. I had to search for a long time. I think that they can be used only few times: insights will rarely be best represented with those kind of charts. Here, I chose a pie chart. Yes, a PIE CHART! Even if billions of people think we should never use them. In this particular case, I believe it was appropriate because I added a clock to make it clear that we are talking about a moment of the day. Not only the pie chart shows part of a whole, but it also represents a specific time of the day. Did I convince you to sometimes use a pie chart? I have to give credit to Udemy for this helpful video. Ok, let's drink a coffee… but only if it is between 9:30 am and 11:30 am !
Interactive viz

Catherine W

I used R to create a circle packed chart showing the breakdown for different weather patterns by season in Seattle. Larger circles indicate more of that weather pattern.
Blog

Chris

Finding a subject was a real challenge, I'm not a big fan of radial charts and forcing something to be radial for the sake of it seemed wrong. I used time in the end because it felt the most natural. Others have commented on how difficult it is to represent a 24hr clock in a single circle - I opted to stick with 24hrs in 360 degrees due to the difficulty of visualising the data any other way. I'm not sure how easy the chart is to interpret but I love the patterns and visual appeal of the result. I think this appeal overrides a lot of the downsides of the radial chart and the difficulties in reading it.
Link

Claire

Although I was intimidated by this month's challenge (I'd never plotted data on a radial chart before), it ended up being a textbook demonstration of the benefits of data viz. I chose a dataset at random (Chicago public transportation ridership for 2017) and plotted the values. They were disappointingly consistent over the course of the year, which made for a pretty boring radial chart. I decided to instead plot the percentage change in ridership (compared to average weekday and weekend values), and I was excited to see that the chart - and the data - were suddenly much more interesting! I could see the outliers, and I could point to them to tell a story about what was happening in Chicago in 2017.
Link

Daniel

I used Tableau to visualize the major killer / victim combinations across the entire Game of Thrones 7 seasons. Each dot represents a unique Killer and victim combination and is sized by the actual number of deaths for this combination.

Dennis

The idea for this visual was born about two years ago at one of my customers. At that time the idea didn't work out the way it was supposed to, but I think it does now. Having data that never takes more than 60 minutes (the thing that went wrong 2 years ago) makes it easy to read the results. To make sure the attention is focused on today I use gray colors for all other parts.
Link

Eddie

A prototype of a radial view to display the sister city connections.

Edwin

I created this radial graph just to show the different eras of Boy Bands throughout the years. Each line on the graph represents a single boy band and the length of the line represents how high their peak song got on the billboard charts. I intentionally distorted the order of the chart to switch up on the coloring of the chart for an aesthetic look but serves no purpose in helping the chart overall.
LINK

Ela

I wanted to find out if there was any correlation between geographical orientation and working hours for European countries in 2018.
Interactive viz

Elvira

I wanted to compare amount as well as categories of my expenses in each month. The data starts from July 2018, that's why January is not on top :)

Franck

It is a remake of a dataviz featured in Visual Capitalist's article Animation: Global Population by Region From 1950 to 2100, which was inspired by geographer Simon Kuestenmacher. For this remake, the main idea was to make it more connected to our Earth (as I found the original pie chart and bar chart lacking this feeling). So I use a disk. The increase of this disk encodes the increase of the global population over years. In this disk, the area of each cell encodes the population of its corresponding region.
Link | Source article

Frans

Short of time creating a brand new radial dataviz for this months #swdchallenge, so I tweaked my 'Does size matter' viz slightly.

Georgios

I find concentric circles fascinating and the first thing that came to mind was to make something that would look like the rings in the Apple Watch activity app.
Link

Hanna

During the #MakeoverMonday with game of thrones data I came up with an idea of creating radial bars - each bar representing a single episode in the series - that I could then place around the throne. In a way the bars would be an extension of the swords that the throne is made of. I'm not sure this qualifies for the challenge as it is not fully radial, but I thought this method fitted perfectly with the topic.

Hesham

This is my first SWD challenge submission. This visualisation made in Tableau and the data preparation was made in Alteryx. It allows the user to interact with the visualisation by clicking on their Zodiac sign, doing so the user get to know the history of their sign and when they are most likely to see their constellation and also how their constellation looks like.
Interactive viz

James

A radial stacked bar chart looking at English Cricket's Greatest batsmen - I love radial charts!
Link

Jared

The inspiration for this viz comes from the many hours I've spent playing Tekken 7 with my daughter during the school holidays. It turns out she's a bit of a natural!
Link

Jerome

My utilization illustrates the number of pickups throughout the year of 2016 based on all 365 days, and the 24 hours within those days.

Johanie

I made my first chord diagram with R for this challenge. I used Statistic Canada data for irrigated area. It's not so easy to play with the aesthetics for this type of graph because it's doesn't work as ggplot but it still possible to do something good. Code is on my blog.

Kate

I was excited at first by this challenge, then after a few days felt uninspired. Lucky for me R.J. Andrews of Info We Trust’s weekly data viz inspiration email hit my inbox on July 3rd, and I found the spark. (If you haven’t signed up for these yet- highly recommend, they’ve all been delightful). R.J. highlighted a chart authored by Thomas Jefferson which displayed the fruit and vegetable availability of a local Washington D.C. farmers market from 1801-1808. I re-designed this into a radial viz and added modern seasonal availability data of the vegetables Jefferson recorded. I had fun learning about some of the veggies that have fallen out of “fashion” and playing around with 19th-century design elements. I write more about my lessons learned in my blog.

Lance

I found this challenge particularly difficult. I am not sure what other platforms are like creating radial charts but I use Tableau and I found creating this type of chart in Tableau a little convoluted! While I appreciate my entry is not as aesthetically pleasing as pretty much all of the examples provided, as a first attempt I am relatively happy. Despite the difficulties it was a great learning experience completing this challenge and well that's what this process is all about!

Leah

I believe time data plots well in a radial graph representing a 24-hour analog clock. I used Seattle collision data to create a high-level view of accident trends throughout the day. I experimented quite a bit with use of color, finally deciding to let the graph (and white space) speak for itself. This plot was created using R.

Ligia

The chart was done on PowerPoint to demonstrate that the data visualization techniques can be used in all tools all the time. The charts show in a simple way how unemployment come impacting the Brazilian people.
Link

Lisa

For this month's challenge, I was fortunate to find quite a few helpful blog posts and video tutorials on building radial charts in Tableau. As I was getting happily lost in the learning process, time was running out, so I kept the challenge simple by creating a radial bar chart. I chose to tell a clear story about the inequality of life expectancy of women around the world. Although not a chart I typically use, I can now see how radial bar charts can work effectively to catch the eye and attention of an audience.
Link

Liz

I had done this type of visualization only once before, so it took me some time to find data that would fit. I decided to focus on international travel since I recently traveled to France and the Netherlands. I found this information from the US Travel Association and used Tableau to make the visualization.

Lori

For this month's challenge, I tried to think of data that naturally occur in circles. I thought of a clock face, and then located some data about toddlers' sleep times. I don't really have fancy data viz technology, but I created this in Excel + PowerPoint. If someone had the technology, I think it would be awesome to be able to hover to see the specific sleep and wake times.
Link

Matthew

One of my favorite video games is FIFA. This visualization is a breakdown of my favorite team MN United. I have never used Sunburst charts, and am not a fan of Pie or Donut. This did take me out of my comfort zone.

Michael

This is my first #SWDchallenge. I was crunched for time but wanted to give it a go. I have never produced a radial chart before so a lot of time went into learning a new technique in Tableau. My story telling suffered as a result but excited to continue to improve my SWD skills.

Neil

I wanted to combine visualising music with a radial chart - given the cyclical nature of Pachelbel's canon there are so many ways I could do this. In the end I settle on this small multiple version visualising each of the 28 sections - we see how the motifs change and follow each other in sequence while the basso profundo part never once changes.
Interactive version synced with music | Website

Pris

Heartbreak can feel endless cycle --Wake up, feel, think too much, and repeat. I chose to rework an old side project mapping out how often the heartbroken consulted their modern day seer, Google. Using Google Trends data, I tried to recreate the somber time loop experience using a radial bar chart. Through the accompanying line chart, bar chart and text, I wanted to highlight the spikes in searches occurring throughout the day.

Rahul

This was a chart built by me a year ago. I saw this chart on Tableau first by Adam Crahen (@acrahen) and Robreto Reif (@robertoreif) article helped me to create this chart.
Interactive viz | Data source | Website

Robert

I decided to first look at nice circular graphs and which requirements they would pose for the data. I saw a very nice circular graph in the rawgraphs gallery, made by Frederik Ruys called The rise and flow of political parties. It was so nice that I immediately decided to go with that type of graph. I soon found out that rawgraphs does not offer this as a standard graph. They have a standard bump chart which is horizontal and you have to make it circular yourself. I think in the future rawgraphs will offer it as a standard graph type, it definitely should be. After that I looked through Gapminder for a dataset. As I was kind of in a hurry I decided to go with the first dataset that would meet the requirements posed by the chart, which was the fertility dataset.

Romina

Marvel superheroes stats and info, it is a visualization about the characters.
Website

Samo

Visualization for all on-screen kills across eight seasons of Game of Thrones where I choose a radial chart to communicate seasons total kills and episodes break down.

Sebastián

Website

Simon B

For my radial viz I decided to encode country and winners through a sunburst and then when titles were wine through radial circles, sized by decade titles. My aim was to show when players and countries dominated the Wimbledon Championships.
Interactive viz | Website

Simon R

Used inspiration from a blog by Bora Beran to have a go at a Coxcomb chart showing average monthly minimum and maximum temperatures for the city of London. Quite a steep learning curve and need to go back and drill into all the table calcs to create it but pleased with the look.

Vasa

A makeover of a segment of a visual story done in 2013, this time focusing on time spent online in our learning management system. It aims to show that e-learning helps students engage with the material even after class time.
Link | Data source

Vijaya

Kiva.org is an online crowdfunding platform to extend financial services to poor and financially excluded people around the world. The radial bar is one of the apt charts for visualizing the loan sanctions by the Kiva organization based on the sector of the loan borrowers. This representation of the data gives a bird-eye-view of the most and least sectors that has loan approvals. Data Source Credit: kaggle
LinkedIn

Vinodh

A look at the wildfires in USA from 1980-2018. California is the most wildfire prone state and 90% of it is human triggered.

Yvan

I made this visualization few months ago to track my yearly distance goal, by month and by week. Interactive visualization

Zak

This viz represents daily temperature, precipitation, and severe weather events between June 1, 2017 and June 1, 2019. I downloaded data from the Brackett Creek SNOTEL station in the Bridger Mountains outside of Bozeman, MT. This visualization was created entirely in R.

Click ♥ if you've made it to the bottom—this helps us know that the time it takes to pull this together is worthwhile! Check out the #SWDchallenge page for more. Thanks for reading!

Continue Reading…

Collapse

Read More

Airport runways orient certain directions that correlate with wind direction in the area. It helps planes land and take off more easily. So, when you map runways around the world, you also get wind patterns, which is what Figures did:

Winds circulate around the globe, forming patterns of gigantic proportions. These patterns become part of human culture and are reflected in our architecture. They are hidden designs, mapping the complexion of the earth, which we can uncover. By orienting on the direction of general winds, airports recreate wind patterns, forming a representation of a global wind map with steel and stone, thus making the invisible visible.

Tags: airport, correlation, wind

Continue Reading…

Collapse

Read More

This post summarizes the interesting aspects of the Day One of the SciPy 2019 lightning talks, a flash round of a dozen ~3 minute talks covering a wide variety of topics.

Continue Reading…

Collapse

Read More

Also: Dealing with categorical features in machine learning; Computer Vision for Beginners: Part 1; Big Data for Insurance; A Summary of DeepMind's Protein Folding Upset at CASP13; The Hackathon Guide for Aspiring Data Scientists

Continue Reading…

Collapse

Read More

As a blogger and curriculum developer, I am often writing about R from a text editor instead of RStudio. I feared that working outside RStudio meant saying goodbye to fantastic hotkeys it provides. Isn’t there a way, I wondered, to set up a keyboard shortcut on my computer so that, for example, Alt + - would always return <- ?

I put out the call on LinkedIn (Let’s connect if we haven’t!), and fortunately my friend Mike Cantrell (a new data analyst himself) had the answer: a text expander. 

I consider myself a productivity geek, but this is a new application to me. A favorite of lawyers and technical writers, text expanders allow you to create custom “hotkeys” of varying length and complexity. Turning tcby into The Country's Best Yogurt, for example, is a nice early 90’s example of what text expanders can do.

New to R? Get your course checklist here.

But, readers, I know you’re more hungry for R productivity than frozen yogurt, so below I’ll walk through how to set up the assignment operator and pipe shortcuts instead. 

1. Download PhraseExpress

A bit of research showed a rather competitive market for text expanders, and I landed on PhraseExpress because it’s free for personal use and works on a variety of platforms and operating systems.  

2. Add a new phrase

The PhraseExpress interface is fairly straightforward, at least for the very basic task we are performing. On the main menu, click Phrases > New Phrase.

3. Describe and assign the shortcuts

By default, PhraseExpress will drop your Description attributes into Phrase content. That’s not the best option for our example: instead, complete the former with R assignment operator and the latter with <- . Be sure to include spaces before and after the assignment operator. 

Now you can assign this phrase to the Alt + - hotkey in the bottom window of this screen: 

One more while we’re here: Let’s make one for %>%, the “pipe” operator. Its RStudio shortcut is Ctrl + Shift + M.  

4. Code away! 

Adding these to PhraseExpress, we can now use these shortcuts anywhere on our device.

Where else to practice this but in the console of my very own DataCamp course?

5. Watch for conflicts

Keep in mind that these shortcuts will follow to all applications on your computer. So, you will run into conflict with any programs using Alt + - or Ctrl + Shift + M as keyboard shortcuts. 

Have you used text expanders before? If so, how? What advice would you give to new users? 

Ready to take your R journey? Check out my comprehensive course, R Explained for Excel Users.

Continue Reading…

Collapse

Read More

Contextual Bilateral Loss (CoBi)
This paper shows that when applying machine learning to digital zoom for photography, it is beneficial to use real, RAW sensor data for training. Existing learning-based super-resolution methods do not use real sensor data, instead operating on RGB images. In practice, these approaches result in loss of detail and accuracy in their digitally zoomed output when zooming in on distant image regions. We also show that synthesizing sensor data by resampling high-resolution RGB images is an oversimplified approximation of real sensor data and noise, resulting in worse image quality. The key barrier to using real sensor data for training is that ground truth high-resolution imagery is missing. We show how to obtain the ground-truth data with optically zoomed images and contribute a dataset, SR-RAW, for real-world computational zoom. We use SR-RAW to train a deep network with a novel contextual bilateral loss (CoBi) that delivers critical robustness to mild misalignment in input-output image pairs. The trained network achieves state-of-the-art performance in 4X and 8X computational zoom. …

Uncertainty-Aware Feature Selection (UAFS)
Missing data are a concern in many real world data sets and imputation methods are often needed to estimate the values of missing data, but data sets with excessive missingness and high dimensionality challenge most approaches to imputation. Here we show that appropriate feature selection can be an effective preprocessing step for imputation, allowing for more accurate imputation and subsequent model predictions. The key feature of this preprocessing is that it incorporates uncertainty: by accounting for uncertainty due to missingness when selecting features we can reduce the degree of missingness while also limiting the number of uninformative features being used to make predictive models. We introduce a method to perform uncertainty-aware feature selection (UAFS), provide a theoretical motivation, and test UAFS on both real and synthetic problems, demonstrating that across a variety of data sets and levels of missingness we can improve the accuracy of imputations. Improved imputation due to UAFS also results in improved prediction accuracy when performing supervised learning using these imputed data sets. Our UAFS method is general and can be fruitfully coupled with a variety of imputation methods. …

Deep Feature Fusion-Audio and Text Modal Fusion (DFF-ATMF)
Sentiment analysis research has been rapidly developing in the last decade and has attracted widespread attention from academia and industry, most of which is based on text. However, the information in the real world usually comes as different modalities. In this paper, we consider the task of Multimodal Sentiment Analysis, using Audio and Text Modalities, proposed a novel fusion strategy including Multi-Feature Fusion and Multi-Modality Fusion to improve the accuracy of Audio-Text Sentiment Analysis. We call this the Deep Feature Fusion-Audio and Text Modal Fusion (DFF-ATMF) model, and the features learned from it are complementary to each other and robust. Experiments with the CMU-MOSI corpus and the recently released CMU-MOSEI corpus for Youtube video sentiment analysis show the very competitive results of our proposed model. Surprisingly, our method also achieved the state-of-the-art results in the IEMOCAP dataset, indicating that our proposed fusion strategy is also extremely generalization ability to Multimodal Emotion Recognition. …

Distance Metric Learned Collaborative Representation Classifier (DML-CRC)
Any generic deep machine learning algorithm is essentially a function fitting exercise, where the network tunes its weights and parameters to learn discriminatory features by minimizing some cost function. Though the network tries to learn the optimal feature space, it seldom tries to learn an optimal distance metric in the cost function, and hence misses out on an additional layer of abstraction. We present a simple effective way of achieving this by learning a generic Mahalanabis distance in a collaborative loss function in an end-to-end fashion with any standard convolutional network as the feature learner. The proposed method DML-CRC gives state-of-the-art performance on benchmark fine-grained classification datasets CUB Birds, Oxford Flowers and Oxford-IIIT Pets using the VGG-19 deep network. The method is network agnostic and can be used for any similar classification tasks. …

Continue Reading…

Collapse

Read More

Julia Lane, NYU Professor, Economist and cofounder of the Coleridge Initiative, presented “Where’s the Data: A New Approach to Social Science Search & Discovery” at Rev. Lane described the approach that the Coleridge Initiative is taking to address the science reproducibility challenge. The approach is to provide remote access for government analysts and researchers to confidential data in a secure data facility and to build analytical capacity and collaborations through an Applied Data Analytics training program.  This article provides a distilled summary and a written transcript of Lane’s talk at Rev. Many thanks to Julia Lane for providing feedback on this post prior to publication.

Session Summary

Science is facing a research reproducibility challenge that hampers data scientists’ and researchers’ ability to accelerate their work and provide insights that impact their organizations. Since Domino’s inception, we have tackled the reproducibility problem to support our customers via continued updates to the platform’s collaboration functionality as well as contributing to the overall public discourse on this blog and at industry events including Rev. In the Rev session, “Where’s the Data: A New Approach to Social Science Search & Discovery”, Julia Lane provided insights into how the Coleridge Initiative is addressing the reproducibility challenge by providing secure remote access to confidential data for government analysts and researchers and to build analytical capacity and collaborations through an Applied Data Analytics training program. The goal is to enable government agencies to make better evidence-based policy decisions with better quality data. Lane, staying true to the ethos of reproducibility, covered how the approach could be used to allow approved analysts to reuse and ingest insights to accelerate their work. Lane discussed the questions Coleridge sought to answer; to improve the analytical rigor associated with working with linked data; how Coleridge built and tested a set of tools that could be applied to identify what data have been used to address different research questions; and how the approach can be used to inform new researchers as they access the secure environment with new projects. Lane closed the session with how the intention of initiative “is to build evidence-based policy making whereby we get better knowledge, get better policy, get resources allocated better, and reduce the cost and burden of collecting information.”

A few highlights from the session include

• how modern approaches can be used to obtain better quality data, and at a lower cost, to help support evidence-based policy decisions.
• unique challenges of sharing confidential government microdata and the importance of access in generating high quality inference
• pragmatic evaluation of risk-utility tradeoffs, or the tradeoff in risk resulting from greater data usage versus the risk of disclosure or reidentification
• a discussion of how Coleridge used the training classes to build the capacity of government agency staff to address data quality challenges
• the importance of pairing training for researchers with access in a secure environment.

Additional insights are available in the written transcript.

Transcript

Julia Lane:

I’m an economist by training and that is where my focus is going to be. Let me tell you the story. I’m at NYU, as you can tell by my strong New York accent, and I’ve spent most of my career working with federal statistical agencies, the agencies that bring you the Decennial Census, the unemployment rate, GDP, and so on. A major challenge that you face with using those kinds of data for decision making is data is very expensive to collect and the quality of the data is going down.

How many of you are familiar with the fact that the Decennial Census is going to be fielded next year? Okay. Roughly how many people do you think there are in the United States? 350, 360 million, we’ll find out more maybe next year. How much do you think it costs to collect…count that number of people? $360 million. Try north of that. $2 billion, do I hear two? Okay. Anyone want to go higher? Higher, more, more. $17 billion, okay, to count the number of people and ask them 10, maybe 11 questions. Right? The challenge that we have is that the quality of the data is going down. People aren’t responding, they’re giving bad responses, and so on.

About three years ago, Evidence-Based Policymaking commission was established, which brought together experts from around the country to figure out how can you develop data to make decisions at better quality and lower cost,. They came up with a set of recommendations, that got passed into the Evidence-Based Policymaking Act this year.

They asked NYU to build a s to build an environment to inform the deliberations of the Commission, and show how you can bring together data from multiple sources and make sense of them at the level of quality that you need to be able to allocate resources to make decisions. That’s why it’s called evidence-based policy.

Of course, the challenge with bringing data together is that the data, particularly on human beings, is quite complicated. When it is generated from multiple different sources, it’s not just a matter of mashing them together. You need to understand how they were generated, what the issues are and so on. Together with that, we built training classes to train government agency staff and researchers and analysts who want to work with the data what the issues…how to work with them.

Here’s the challenge. When you’re working with complicated data and when it is confidential, because it is data on human beings, it’s very difficult to find out what work has been done on them before. Every time someone starts working with the data, it’s a tabula rasa. You can’t figure out what’s going on. The challenge that we had to solve is the challenge I’m going to talk to you about today, which is how do you find, when you land on a dataset that is newly generated, that isn’t curated in any way …. how do I figure out what’s in there and how do I figure out who else has worked with it? And if you think about it, this is an amazon.com problem, right?

The reason Jeff Bezos made so much money is that he solved the problem of figuring out what was in books, figuring out by information that was generated by the way other people who’ve use the book rather than just the way that people had produced the book. What we wanted to do was to build an amazon.com for data, and that’s the story I’m going to talk about today, and give you some sense about the platform that we’re trying to build.

What am I talking about with the new types of data? Back in the day, data used to be generated by someone filling out a survey form, right? A statistical agency collected it, curated it, documented it, sent it out. Or it could be administrative records, records that are generated from the administration of a government program like tax data. Nowadays, we’re also looking at the new types of data, dataset generated by sensors, generated by your card swipes, like retail trade data, Mastercard or IRI data or your DNA. These are complicated datasets. They don’t come nicely curated, but they can add a lot of understanding about how to allocate policy.

That data needs to be shared. The challenge that we face is when it’s confidential data on human beings, there are, quite rightly, many prohibitions on sharing the knowledge. Everything is much more siloed than in the open data world that you’re used to dealing with. For example, as this slide shows, the commissioners in the city of Baltimore will get together every time a child dies to share information about all the government programs that they’ve touched. It might be housing, education, welfare, foster care and so on. But the only time they share the knowledge is when the kid is dead. What we’re trying to do here is to build a knowledge infrastructure that enables us to share the knowledge before children die in ways that can improve policy.

But the challenge is, and this is a risk-utility tradeoff, that the value of working with confidential data is that the more people and the more use it gets, the better off the policy is, but also the greater the risk of disclosure. What you have to do is you have to try and manage that disclosure. If you can, you really can build better policy. New Zealand is a good exampleAs these slides, developed by the former prime minister, Bill English, show. Like most countries or cities, you know, there’s three big areas of expenditure: education, health, pensions. And what you want to do, if you want to allocate resources a little bit better, is to use the integrated data generated from multiple government programs a little bit better.

What do I mean by integrated data, for example? Here’s a kid, and the age of the kid is along here, down the bottom here is the cost to the taxpayer, not just other things. You look at how this kid hits Children, Youth, Family Services, Abuse, Foster Care, Education, Youth Justice, Income Support or Welfare, right? Kid gets born, by age about two and a half showing up with Children, Youth, Family Services and notifications of abuse. You start seeing the kid here, more abuse, more visits from Family Services. Starts education, by about 9, 10, 11, spotty education, gets taken into care then by 17, he hits into the Youth Justice, and then goes into income support. Pretty predictable if you put that information together. I don’t need to belabor the issues that are associated with it here.

That kind of information, if you put it together and understand it, can help allocate resources. This is getting a school certificate, education qualification. If the kid gets it by age 18, now, in pretty good shape for the future. If they don’t get it, it’s a pretty bad indicator. You can rank, based on the data, you can rank the likelihood of a kid achieving, not achieving this school certificate by age 18, and you can figure out who are the kids who are at the highest risk of not getting the qualification. If you allocate resources away from, you know, kids like my kids, they don’t need interventions. They don’t need the kind of services that these kids need. You can reallocate funding and you can tremendously reduce the cost to both the taxpayer and transform those children’s lives.

It is this type of vision that led to the Evidence-Based Policymaking Act. The big thing is, is how do you put the data together securely, in a clearinghouse? That the kind of work that I’m talking about can be implemented by government analysts and by researchers in a secure way so that the risk of re-identification is minimized.

We built a clearinghouse. which we called the Administrative Data Research Facility. I don’t like the term clearinghouse for any number of reasons. It has to be program…mission specific, so I prefer the term facility. But the key thing was not just building the clearinghouse but also building a training program that worked with it.

I’m not going to go into too much detail about it, but the basic idea is you’re going to have a secure environment. And then of course, a major challenge is that you have to have telemetry to figure out who’s accessing it. But you also need to have metadata around the data. You need to have a rich context because if it’s just zeros and ones, I have no clue what it is. And many of you who worked with open data will have observed that the open data can be a bit dodgy in terms of quality. Part of the challenge is the way in which the data was generated, it was just kind of…there’s vomit of data that was put together and summarized and doesn’t really have any cables going back to the microdata engine. And what you really need is you want the data users to be to be involved in the metadata documentation.

In other words, again, drawing the analogy with the statistical system, the way in which the agencies generate data for use as human beings who create metadata documentation very painstakingly, and you get a report on what all the variables mean, how they were generated and so on. Highly manual process. What we want to do is we wanted to generate an automated way of finding out what information is in the data and what’s the quality of that information. Just let me give you a flavor of what those is. In one of the classes we have data on ex-offenders. The programmatic question is, is what’s the impact of access to jobs and neighborhood characteristics on the earnings and employment outcomes of ex-offenders, tand their subsequent recidivism of.

Here’s where we’re in…why this is all on [inaudible 00:14:09], They’re going back and forth. Wouldn’t it be great to have that tacit knowledge codified so that as people start working with the data, the metadata documentation is generated automatically, like amazon.com. Right? That’s the basic idea. You know, instead of you saying, “Where’s the data coming from and how was it documented just based on the way it was produced,” you’ve got the community telling you something about what the data is about.

Okay. Here’s my challenge. I want to figure out, when I land on a dataset, who else has worked with the data on what topics and for what results. And then I want to generate a community that’s going to contribute knowledge. It’s kind of an amazon.com for data.

Okay. How am I going to build a machine that’s going to do that? Remember the statistical agencies, and I slammed them at the beginning, but, you know, these are great people. These are hardworking, wonderful human beings who have great motivation, but they’re like a pre-Industrial Revolution data factory. Now what we’re trying to do is we’re trying to build a modern data factory, a modern approach to automate the generation of the metadata.

Essentially, what we’re going to try and figure out how to do is we are going to scope the question, pose it to the computer science community, natural language processing, machine learning community, and say, “Can you figure out how you can learn from it, automate it, rinse and repeat?” And here’s the core insight. We’re interested in who has worked with this data before, and identify them and then figure out what they did with it. If you think about it, all of that knowledge is embedded in publications, either published work or working papers or government reports. It’s in a document somewhere. In that document, if it’s empirical, someone has said, “Here’s my question, here’s what I’m going to do with it, and here’s the section that describes the data.”

What I want to do is I want to tee the computer scientists up to help me figure out where is the dataset and where the semantic context that’s going to point me to that. Okay. And then I’m going to get the community at large to tell me whether they’ve done it right or wrong, and then fix it from there.

Essentially, one of the communities we’re working with U.S. Department of Agriculture. But anyway, USDA, you’ll see one of the things they look at is NHANES, the nutrition education dataset. You’ll see here, here they have something that says analytical sample. They say something about the data. What we want is for the computer scientists to go figure out where that data is.

That’s what we did. We ran a competition. We took a hand-curated corpus. Social science data repositories, public use ones, sit in different places across the country. One is at University of Michigan, you have ICPSR. There are three people who, every day, their job is to read papers and say which one of the ICPSR datasets is in that corpus, manually. Then they write it down and then they put it up to say what’s been done. We took that corpus and we ran a competition. We had 20 teams from around the world compete. Twelve of them submitted code, four very kind people here helped advertise it, thank you for that. Then we had four finalists. The model that actually enable was…that we were amazed was they could actually do this.

Think about it. If I hold up a publication and it… Can you tell me what the dataset is that’s referenced in there? And the answer is, of course, no. The baseline is zero. What the winning algorithm did, it correctly identified the dataset that was being cited in the publication 54% of the time. And that’s amazing, right?

Now there’s a lot of work to be done yet. I’ve skipped over all the bugs and problems and on, but it’s still super encouraging, right? Because now, once I get that dataset to publication link, that gives me the rich context, that gives me the potential to find out everything else because there’s a lot of work that’s done on publications by my colleagues at Digital Science, UberResearch. They have linked to publications over the past 10 years: grants, policy documents, patents, clinical trials, and so on. Once I get that dyad, I’m off to the races, right? What that enables me to do is to figure out, for a particular publication, everything around it. That was my goal.

Now there’s a lot of work that needs to be done around that. But, for example, again, this is a Dimensions website, and I’m not going to have time to go live because I’m going to get yelled at, but I could show you live, you can type in the dataset, and you get lots of related information. You find out who the researchers are, what the related topics are and so on. And that is going to give me the knowledge that I was looking for. It was a pretty buggy model, even though it was amazing. There’s, you know, whole amount of work that needs to be done on it.

Now what we want to do is to go and get that dyad clean up. The biggest problem is, and you probably already figured this out, but the search that we had was on titled datasets. It’s things that were called American Community Survey or NHANES or PSID or something like that. Where there’s a lot of datasets that human beings work with, they’ll say, “Oh, we were working with LinkedIn data, we were working with Twitter data,” something that’s not labeled or with retail IRI, retail scanner data, it doesn’t actually have a title that you could go and find. We need much better knowledge from the semantic context. That means we need to develop a corpus, a tagged corpus that the machine learning algorithms can be trained on.

We’re working with Dimensions in Digital Science, and we’re also trying to get human-curated input in a number of different ways. One is working with publishers where the [inaudible 00:22:01], when an author submits a publication and they say, “Can you give us some keywords,” we can tell them, “Give us some…tell us what datasets are in there.” Right? If they just tell us what datasets are in there, I’ve got a dyad right away. Right? And then I can train.

When researchers are getting onboarded into a secure environment and then they are asking about what datasets are available, you could get them to contribute their knowledge as well. In the classes, we run 300 government analysts through who are subject matter experts, we could get them to tell us what they know about datasets that are available for the common good. If any of you are social scientists, please go to this and we’re asking our colleagues to just fill it. It turns out, you know, if we get a thousand well curated document with public datasets, that’s going to be enough to see the next iteration.

This is where we go next. The Digital Science guys of the guys who brought us Altmetrics. It turns out that people really like…they go to that shiny little badge, the Altmetrics badge, and click on it. What we’re designing is if you type in now and look for a dataset, and we’re working with the Deutsche Bundesbank as well, it will then pull up the related publications. Then the idea here is every publication that’s pulled up, you get the dataset context that’s in there and it’s going to say how many experts, how many papers, how many code books, how many annotations there are associated with it. Then when you click on that, up pops more rich context, I can find all the papers, the other papers, experts, code books, annotations and related datasets. And then up here is a call for action, right? We don’t have to have everyone responding, but we’ll trial this out to see how well we do on that. And then of course, feed that into this approach.

Then the last step, and this is work with Brian Granger and Fernando Perez, is to build it into Jupyter notebooks. One of the things that Fernando and Brian have been doing is they’re trying to make the workbooks more collaborative. Because right now, it’s just a single computational narrative. And they’re also trying to work with them, to be able to work with confidential microdata for all the reasons that I talked about.

Here’s the basic idea. Currently, now, when you land in on a dataset, if you’re lucky, all you get is data that is generated by the way the dataset was produced. The analogy example is Jeff Bezos again. When you look for a book, what do you find? The ISBN number, the author, the title, the publisher, right? That’s the metadata, by the way, the book was produced. What you really want is knowledge about the data itself. I don’t have to go into a bookstore and find out, I can just find out who else like me has used the information. And again, we’re building this into Jupyter notebooks, we, the Jupyter team are, in conjunction with our team, and the notion here is remember the Slack communications that we had, build that in to annotations. Here’s Brian and Fernando just putting stuff in, but build that into the annotation that tacit knowledge gets codified and built into the graph model that underlies the data infrastructure.

That’s the sweep of the story. We want to be able to build evidence-based policymaking whereby we get better knowledge, get better policy, get resources allocated better, and reduce the cost and the burden of collecting information. We started off with building a secure environment and building workforce capacity around it. We’re kind of at this stage right now. Where we want to head is build a platform.

If you want more information, we’re hiring here in scenic New York, at NYU. A lot of the information is here and also on our website. You may wonder why it’s called the Coleridge Initiative. How many of you have heard of Samuel Taylor Coleridge? Great. Okay. Very famous for “The Rime of the Ancient Mariner,” right? We were trying to figure out what to call this thing, and we thought, “Data Science for the Public Good,” [blech] “Evidence-Based Policy,” [blech]. Coleridge Initiative seemed obvious, right? Rime of the Ancient Mariner, “Water, water everywhere, nor any drop to drink,” right? Here, it’s “Data, data everywhere, we have to stop and think.” That’s why it’s called the Coleridge Initiative.

^{This transcript has been edited for readability.}

Continue Reading…

Collapse

Read More

“And that’s where the statistician needs to take it easy:
• Start with the results, so the audience has a clear view on the outcome
• Proceed to explain the analysis simply and with a minimum of statistical jargon
• Describe what an algorithm does, not the specifics of your killer algo
• Visualize the inputs (e.g.: a correlation matrix showing an ‘influence heat map’)
• Visualize the process (e.g.: a regression line on a chief predictor variable)
• Visualize the results (e.g.: a lift chart to show how much the analysis is improving results)
• Always, always tie each step back to the business challenge
• Always be open to questions and feedback.” Andrew Pease ( November 3, 2014 )

Continue Reading…

Collapse

Read More

In the recent issue of Madolyn Smith’s Conversations with Data newsletter hosted by DataJournalism.com, she discusses “bad charts,” featuring submissions from several dataviz bloggers, including myself.

What is a “bad chart”? Based on this collection of curated "bad charts", it is not easy to nail down “bad-ness”. The common theme is the mismatch between the message intended by the designer and the message received by the reader, a classic error of communication. How such mismatch arises depends on the specific example. I am able to divide the “bad charts” into two groups: charts that are misinterpreted, and charts that are misleading.

Charts that are misinterpreted

The Causes of Death entry, submitted by Alberto Cairo, is a “well-designed” chart that requires “reading the story where it is inserted and the numerous caveats.” So readers may misinterpret the chart if they do not also partake the story at Our World in Data which runs over 1,500 words not including the appendix.

The map of Canada, submitted by Highsoft, highlights in green the provinces where the majority of residents are members of the First Nations. The “bad” is that readers may incorrectly “infer that a sizable part of the Canadian population is First Nations.”

In these two examples, the graphic is considered adequate and yet the reader fails to glean the message intended by the designer.

Charts that are misleading

Two fellow bloggers, Cole Knaflic and Jon Schwabish, offer the advice to start bars at zero (here's my take on this rule). The “bad” is the distortion introduced when encoding the data into the visual elements.

The Color-blindness pictogram, submitted by Severino Ribecca, commits a similar faux pas. To compare the rates among men and women, the pictograms should use the same baseline.

In these examples, readers who correctly read the charts nonetheless leave with the wrong message. (We assume the designer does not intend to distort the data.) The readers misinterpret the data without misinterpreting the graphics.

Using the Trifecta Checkup

In the Trifecta Checkup framework, these problems are second-level problems, represented by the green arrows linking up the three corners. (Click here to learn more about using the Trifecta Checkup.)

The visual design of the Causes of Death chart is not under question, and the intended message of the author is clearly articulated in the text. Our concern is that the reader must go outside the graphic to learn the full message. This suggests a problem related to the syncing between the visual design and the message (the QV edge).

By contrast, in the Color Blindness graphic, the data are not under question, nor is the use of pictograms. Our concern is how the data got turned into figurines. This suggests a problem related to the syncing between the data and the visual (the DV edge).

***

When you complain about a misleading chart, or a chart being misinterpreted, what do you really mean? Is it a visual design problem? a data problem? Or is it a syncing problem between two components?

Continue Reading…

Collapse

Read More

• Object-Capability as a Means of Permission and Authority in Software Systems
• A Scalable Framework for Multilevel Streaming Data Analytics using Deep Learning
• Mutual Reinforcement Learning
• Logic Conditionals, Supervenience, and Selection Tasks
• Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning
• Evaluating Explanation Without Ground Truth in Interpretable Machine Learning
• A Self-Attentive model for Knowledge Tracing
• Deep Social Collaborative Filtering
• Meta-Learning for Black-box Optimization
• Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches
• Quantum Data Fitting Algorithm for Non-sparse Matrices
• Perception of visual numerosity in humans and machines
• Selection Heuristics on Semantic Genetic Programming for Classification Problems
• Natural Adversarial Examples
• Mediation Challenges and Socio-Technical Gaps for Explainable Deep Learning Applications
• Construction and enumeration for self-dual cyclic codes of even length over $\mathbb{F}_{2^m} + u\mathbb{F}_{2^m}$
• On the Relationships Between Average Channel Capacity, Average Bit Error Rate, Outage probability and Outage Capacity over Additive White Gaussian Noise Channels
• The Bach Doodle: Approachable music composition with machine learning at scale
• Topology Based Scalable Graph Kernels
• Boosting Resolution and Recovering Texture of micro-CT Images with Deep Learning
• Adaptive Flux-Only Least-Squares Finite Element Methods for Linear Transport Equations
• DeepRace: Finding Data Race Bugs via Deep Learning
• Integrating the Data Augmentation Scheme with Various Classifiers for Acoustic Scene Modeling
• On Convergence and Optimality of Best-Response Learning with Policy Types in Multiagent Systems
• Geometric Convergence of Distributed Gradient Play in Games with Unconstrained Action Sets
• A Short Note on the Kinetics-700 Human Action Dataset
• A portable potentiometric electronic tongue leveraging smartphone and cloud platforms
• Concentration of the matrix-valued minimum mean-square error in optimal Bayesian inference
• Cataloging Accreted Stars within Gaia DR2 using Deep Learning
• Linear Receivers for Massive MIMO Systems with One-Bit ADCs
• Slow Feature Analysis for Human Action Recognition
• Robust Variational Autoencoders for Outlier Detection in Mixed-Type Data
• Quant GANs: Deep Generation of Financial Time Series
• Towards Near-imperceptible Steganographic Text
• Period mimicry: A note on the $(-1)$-evaluation of the peak polynomials
• A row-invariant parameterized algorithm for integer programming
• Sampled-Data Observers for Delay Systems and Hyperbolic PDE-ODE Loops
• Comparison Between Algebraic and Matrix-free Geometric Multigrid for a Stokes Problem on Adaptive Meshes with Variable Viscosity
• CupQ: A New Clinical Literature Search Engine
• A Stratification Approach to Partial Dependence for Codependent Variables
• Output maximization container loading problem with time availability constraints
• Imaginary replica analysis of loopy regular random graphs
• PPO Dash: Improving Generalization in Deep Reinforcement Learning
• Towards network admissible optimal dispatch of flexible loads in distribution networks
• MaskPlus: Improving Mask Generation for Instance Segmentation
• Ramanujan Congruences for Fractional Partition Functions
• Asymptotic stabilization of a system of coupled $n$th–order differential equations with potentially unbounded high-frequency oscillating perturbations
• DOD-ETL: Distributed On-Demand ETL for Near Real-Time Business Intelligence
• Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs
• Deep learning-based color holographic microscopy
• Overcoming the curse of dimensionality in the numerical approximation of Allen-Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations
• Lower Bounding the AND-OR Tree via Symmetrization
• Padé Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networks
• Condensed Ricci Curvature of Complete and Strongly Regular Graphs
• Defining mediation effects for multiple mediators using the concept of the target randomized trial
• Real-time Hair Segmentation and Recoloring on Mobile GPUs
• Binary Decision Diagrams: from Tree Compaction to Sampling
• Almost all Steiner triple systems are almost resolvable
• Low-supervision urgency detection and transfer in short crisis messages
• A Data-Driven Game-Theoretic Approach for Behind-the-Meter PV Generation Disaggregation
• Designing Perfect Simulation Algorithms using Local Correctness
• Development of a General Momentum Exchange Devices Fault Model for Spacecraft Fault-Tolerant Control System Design
• Independence numbers of Johnson-type graphs
• AugLabel: Exploiting Word Representations to Augment Labels for Face Attribute Classification
• Elastic depths for detecting shape anomalies in functional data
• Some error estimates for the DEC method in the plane
• High-order couplings in geometric complex networks of neurons
• Partitioning Graphs for the Cloud using Reinforcement Learning
• Increasing Power for Observational Studies of Aberrant Response: An Adaptive Approach
• Subspace Determination through Local Intrinsic Dimensional Decomposition: Theory and Experimentation
• Efficient Pipeline for Camera Trap Image Review
• Hands Off my Database: Ransomware Detection in Databases through Dynamic Analysis of Query Sequences
• Improving 3D Object Detection for Pedestrians with Virtual Multi-View Synthesis Orientation Estimation
• Nonlinear filtering of stochastic differential equations driven by correlated Lévy noises
• Rethinking RGB-D Salient Object Detection: Models, Datasets, and Large-Scale Benchmarks
• AR(1) processes driven by second-chaos white noise: Berry-Esséen bounds for quadratic variation and parameter estimation
• A simplified proof of CLT for convex bodies
• Some Black-box Reductions for Objective-robust Discrete Optimization Problems Based on their LP-Relaxations
• Planar graphs without 7-cycles and butterflies are DP-4-colorable
• Study of Max-Link Relay Selection with Buffers for Multi-Way Cooperative Multi-Antenna Systems
• 2nd Place Solution to the GQA Challenge 2019
• Efficient Autonomy Validation in Simulation with Adaptive Stress Testing
• Instant Motion Tracking and Its Applications to Augmented Reality
• Asynchronous Coded Caching
• A Bird’s Eye View of Nonlinear System Identification
• Ethical Underpinnings in the Design and Management of ICT Projects
• Alternating Dynamic Programming for Multiple Epidemic Change-Point Estimation
• Stochastic viscosity solutions for stochastic integral-partial differential equations and singular stochastic control
• Hydrodynamic synchronization and collective dynamics of colloidal particles driven along a circular path
• A Quantum-inspired Algorithm for General Minimum Conical Hull Problems
• Energy-efficient Alternating Iterative Secure Structure of Maximizing Secrecy Rate for Directional Modulation Networks
• Stereo-based terrain traversability analysis using normal-based segmentation and superpixel surface analysis
• EL-Shelling on Comodernistic Lattices
• Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving
• Automated Deobfuscation of Android Native Binary Code
• CL-Shellable Posets with No EL-Shellings
• Noise Removal of FTIR Hyperspectral Images via MMSE
• An Inter-Layer Weight Prediction and Quantization for Deep Neural Networks based on a Smoothly Varying Weight Hypothesis
• Quality-aware skill translation models for expert finding on StackOverflow
• Improved Reinforcement Learning through Imitation Learning Pretraining Towards Image-based Autonomous Driving
• The Quantum Version Of Classification Decision Tree Constructing Algorithm C5.0
• Deep inspection: an electrical distribution pole parts study via deep neural networks
• The continuous Bernoulli: fixing a pervasive error in variational autoencoders
• Coherency and Online Signal Selection Based Wide Area Control of Wind Integrated Power Grid
• Discontinuous Galerkin Finite Element Methods for the Landau-de Gennes Minimization Problem of Liquid Crystals
• Modeling competitive evolution of multiple languages
• Vibrational spectrum derived from the local mechanical response in disordered solids
• AirwayNet: A Voxel-Connectivity Aware Approach for Accurate Airway Segmentation Using Convolutional Neural Networks
• Broadcast Distributed Voting Algorithm in Population Protocols
• Quantifying replicability and consistency in systematic reviews
• Labelings vs. Embeddings: On Distributed Representations of Distances
• A generic rule-based system for clinical trial patient selection
• The Impact of Tribalism on Social Welfare
• Distributed data storage for modern astroparticle physics experiments
• Light Multi-segment Activation for Model Compression
• Global and local pointwise error estimates for finite element approximations to the Stokes problem on convex polyhedra
• Separable Convolutional LSTMs for Faster Video Segmentation
• Cascade RetinaNet: Maintaining Consistency for Single-Stage Object Detection
• Learning Depth from Monocular Videos Using Synthetic Data: A Temporally-Consistent Domain Adaptation Approach
• Minimal-norm static feedbacks using dissipative Hamiltonian matrices
• Deep Reinforcement Learning Based Robot Arm Manipulation with Efficient Training Data through Simulation
• A Unified Framework for Problems on Guessing, Source Coding and Task Partitioning
• A General Framework for Uncertainty Estimation in Deep Learning
• Modeling User Selection in Quality Diversity
• Partial Solvers for Generalized Parity Games
• Mango Tree Net — A fully convolutional network for semantic segmentation and individual crown detection of mango trees
• Single-bit-per-weight deep convolutional neural networks without batch-normalization layers for embedded systems
• Human Pose Estimation for Real-World Crowded Scenarios
• The Bregman-Tweedie Classification Model
• Assessing Refugees’ Integration via Spatio-temporal Similarities of Mobility and Calling Behaviors
• Performance Assessment of Kron Reduction in the Numerical Analysis of Polyphase Power Systems
• A theorem about partitioning consecutive numbers
• Improving Bayesian Local Spatial Models in Large Data Sets
• On the $L_p$-error of the Grenander-type estimator in the Cox model
• Graphs with large girth and free groups
• Semi-supervised Breast Lesion Detection in Ultrasound Video Based on Temporal Coherence
• Machine learning without a feature set for detecting bursts in the EEG of preterm infants
• Language comparison via network topology
• A Subjective Interestingness measure for Business Intelligence explorations
• Abstract categorial grammars with island constraints and effective decidability
• on removal of perfect matching from folded hypercubes
• Fused Detection of Retinal Biomarkers in OCT Volumes
• On the Variational Iteration Method for the Nonlinear Volterra Integral Equation
• Stochastic Evolution of spatial populations: From configurations to genealogies and back
• A Unified Deep Framework for Joint 3D Pose Estimation and Action Recognition from a Single RGB Camera
• Random projections and sampling algorithms for clustering of high-dimensional polygonal curves
• Representative Days for Expansion Decisions in Power Systems
• Effect of disorder correlation on Anderson localization of two-dimensional massless pseudospin-1 Dirac particles in a random one-dimensional scalar potential
• Lossless Prioritized Embeddings
• Positive specializations of symmetric Grothendieck polynomials
• Stochastic gradient Markov chain Monte Carlo
• Detecting anomalies in fibre systems using 3-dimensional image data
• Speed estimation evaluation on the KITTI benchmark based on motion and monocular depth information
• X-Net: Brain Stroke Lesion Segmentation Based on Depthwise Separable Convolution and Long-range Dependencies
• Latent Adversarial Defence with Boundary-guided Generation
• Uniqueness and characterization of local minimizers for the interaction energy with mildly repulsive potentials
• CLCI-Net: Cross-Level fusion and Context Inference Networks for Lesion Segmentation of Chronic Stroke
• Gender Balance in Computer Science and Engineering in Italian Universities
• Threshold Logical Clocks for Asynchronous Distributed Coordination and Consensus
• Improving Semantic Segmentation via Dilated Affinity
• Outliers in meta-analysis: an asymmetric trimmed-mean approach
• Applying twice a minimax theorem
• Transmission Power Control for Remote State Estimation in Industrial Wireless Sensor Networks
• Unforeseen Evidence
• Computing Nested Fixpoints in Quasipolynomial Time
• Data Selection for training Semantic Segmentation CNNs with cross-dataset weak supervision
• Morphisms of Skew Hadamard Matrices
• Cayley Structures and Coset Acyclicity
• Adaptive Prior Selection for Repertoire-based Online Learning in Robotics
• Dark Patterns at Scale: Findings from a Crawl of 11K Shopping Websites
• Neural Language Model Based Training Data Augmentation for Weakly Supervised Early Rumor Detection
• Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation
• Structured Variational Inference in Unstable Gaussian Process State Space Models
• Information processing constraints in travel behaviour modelling: A generative learning approach
• Embedded Ridge Approximations: Constructing Ridge Approximations Over Localized Scalar Fields For Improved Simulation-Centric Dimension Reduction
• Pedestrian Tracking by Probabilistic Data Association and Correspondence Embeddings
• Homophily as a process generating social networks: insights from Social Distance Attachment model
• A note on duality theorems in mass transportation
• How much real data do we actually need: Analyzing object detection performance using synthetic and real data
• SGD momentum optimizer with step estimation by online parabola model
• Shrinkage in the Time-Varying Parameter Model Framework Using the R Package shrinkTVP
• RadioTalk: a large-scale corpus of talk radio transcripts
• Prediction of neural network performance by phenotypic modeling
• Anatomically-Informed Multiple Linear Assignment Problems for White Matter Bundle Segmentation
• On The Termination of a Flooding Process
• Data-driven strategies for optimal bicycle network growth
• A Reduced Order technique to study bifurcating phenomena: application to the Gross-Pitaevskii equation
• Variable selection in sparse high-dimensional GLARMA models
• Massive MU-MIMO-OFDM Uplink with Direct RF-Sampling and 1-Bit ADCs
• On the smallest singular value of multivariate Vandermonde matrices with clustered nodes
• Measuring I2P Censorship at a Global Scale
• Two-stage sample robust optimization
• A Two-Stage Approach to Multivariate Linear Regression with Sparsely Mismatched Data
• Step-by-Step Community Detection for Volume-Regular Graphs
• Efficient Segmentation: Learning Downsampling Near Semantic Boundaries
• The Tradeoff Between Privacy and Accuracy in Anomaly Detection Using Federated XGBoost
• On the Performance of Renewable Energy-Powered UAV-Assisted Wireless Communications
• Security Smells in Infrastructure as Code Scripts
• EnforceNet: Monocular Camera Localization in Large Scale Indoor Sparse LiDAR Point Cloud
• Predicting Next-Season Designs on High Fashion Runway
• From Harnack inequality to heat kernel estimates on metric measure spaces and applications
• Explaining Classifiers with Causal Concept Effect (CaCE)
• Fast, Provably convergent IRLS Algorithm for p-norm Linear Regression
• On the ”steerability’ of generative adversarial networks
• Ordinal pattern probabilities for symmetric random walks
• Tightness and tails of the maximum in 3D Ising interfaces
• Hubs and authorities of the scientific migration network

Continue Reading…

Collapse

Read More

A Scalable Framework for Multilevel Streaming Data Analytics using Deep Learning

Mutual Reinforcement Learning

Recently, collaborative robots have begun to train humans to achieve complex tasks, and the mutual information exchange between them can lead to successful robot-human collaborations. In this paper we demonstrate the application and effectiveness of a new approach called \textit{mutual reinforcement learning} (MRL), where both humans and autonomous agents act as reinforcement learners in a skill transfer scenario over continuous communication and feedback. An autonomous agent initially acts as an instructor who can teach a novice human participant complex skills using the MRL strategy. While teaching skills in a physical (block-building) () or simulated (Tetris) environment (), the expert tries to identify appropriate reward channels preferred by each individual and adapts itself accordingly using an exploration-exploitation strategy. These reward channel preferences can identify important behaviors of the human participants, because they may well exercise the same behaviors in similar situations later. In this way, skill transfer takes place between an expert system and a novice human operator. We divided the subject population into three groups and observed the skill transfer phenomenon, analyzing it with Simpson’s psychometric model. 5-point Likert scales were also used to identify the cognitive models of the human participants. We obtained a shared cognitive model which not only improves human cognition but enhances the robot’s cognitive strategy to understand the mental model of its human partners while building a successful robot-human collaborative framework.

Logic Conditionals, Supervenience, and Selection Tasks

Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning

Evaluating Explanation Without Ground Truth in Interpretable Machine Learning

A Self-Attentive model for Knowledge Tracing

Deep Social Collaborative Filtering

Meta-Learning for Black-box Optimization

Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches

Quantum Data Fitting Algorithm for Non-sparse Matrices

We propose a quantum data fitting algorithm for non-sparse matrices, which is based on the Quantum Singular Value Estimation (QSVE) subroutine and a novel efficient method for recovering the signs of eigenvalues. Our algorithm generalizes the quantum data fitting algorithm of Wiebe, Braun, and Lloyd for sparse and well-conditioned matrices by adding a regularization term to avoid the over-fitting problem, which is a very important problem in machine learning. As a result, the algorithm achieves a sparsity-independent runtime of for an dimensional Hermitian matrix , where denotes the condition number of and is the precision parameter. This amounts to a polynomial speedup on the dimension of matrices when compared with the classical data fitting algorithms, and a strictly less than quadratic dependence on .

Perception of visual numerosity in humans and machines

Selection Heuristics on Semantic Genetic Programming for Classification Problems

Natural Adversarial Examples

Mediation Challenges and Socio-Technical Gaps for Explainable Deep Learning Applications

Continue Reading…

Collapse

Read More

Multi-armed bandits a simple but very powerful framework for algorithms that make decisions over time under uncertainty. An enormous body of work has accumulated over the years, covered in several books and surveys. This book provides a more introductory, textbook-like treatment of the subject. Each chapter tackles a particular line of work, providing a self-contained, teachable technical introduction and a review of the more advanced results. The chapters are as follows: Stochastic bandits; Lower bounds; Bayesian Bandits and Thompson Sampling; Lipschitz Bandits; Full Feedback and Adversarial Costs; Adversarial Bandits; Linear Costs and Semi-bandits; Contextual Bandits; Bandits and Zero-Sum Games; Bandits with Knapsacks; Incentivized Exploration and Connections to Mechanism Design. Status of the manuscript: essentially complete (modulo some polishing), except for last chapter, which the author plans to add over the next few months. Introduction to Multi-Armed Bandits

Continue Reading…

Collapse

Read More

R is great for report generation. Shiny allows us to easily create web apps that generate a variety of reports with R.

This post details a demo Shiny app that generates an Excel report, a PowerPoint report, and a PDF report:

The full Shiny app source code is available here. Also, we included a more basic Shiny app that generates an Excel report at the end of this post. Follow up posts will include similar simple Shiny apps generating PowerPoint and PDF reports.

Here are some screenshots of the Shiny app that generates the reports. You click the button in the left sidebar to select the report type (Excel, PowerPoint, or PDF).

And then you click a the report type button to generate the report. Simple as that!

In this app, the generated reports rely on the user selected “Valuation Date” Shiny input (in the left sidebar). Your report generating Shiny app would, of course, include your data and the Shiny inputs necessary for your custom report.

library(shiny)
library(openxlsx)

# create some example data to download
my_table <- data.frame(
  Name = letters[1:4],
  Age = seq(20, 26, 2),
  Occupation = LETTERS[15:18],
  Income = c(50000, 20000, 30000, 45000)
)

# add a totals row
my_table <- rbind(
  my_table,
  data.frame(
    Name = "Total",
    Age = NA_integer_,
    Occupation = "",
    Income = sum(my_table$Income)
  )
)

# minimal Shiny UI
ui <- fluidRow(
  column(
    width = 12,
    align = "center",
    tableOutput("table_out"),
    br(),
    downloadButton(
      "download_excel", 
      "Download Data to Excel"
    )
  )
)

# minimal Shiny server
server <- function(input, output) {
  output$table_out <- renderTable({
    my_table
  })
  
  
  output$download_excel <- downloadHandler(
    filename = function() {
      "employee_data.xlsx"
    },
    content = function(file) {
      my_workbook <- createWorkbook()
      
      addWorksheet(
        wb = my_workbook,
        sheetName = "Employee Data"
      )
      
      setColWidths(
        my_workbook,
        1,
        cols = 1:4,
        widths = c(6, 6, 10, 10)
      )
      
      writeData(
        my_workbook,
        sheet = 1,
        c(
          "Company Name",
          "Employee Data"
        ),
        startRow = 1,
        startCol = 1
      )
      
      addStyle(
        my_workbook,
        sheet = 1,
        style = createStyle(
          fontSize = 24,
          textDecoration = "bold"
        ),
        rows = 1:2,
        cols = 1
      )
      
      writeData(
        my_workbook,
        sheet = 1,
        my_table,
        startRow = 5,
        startCol = 1
      )
      
      addStyle(
        my_workbook,
        sheet = 1,
        style = createStyle(
          fgFill = "#1a5bc4",
          halign = "center",
          fontColour = "#ffffff"
        ),
        rows = 5,
        cols = 1:4,
        gridExpand = TRUE
      )
      
      addStyle(
        my_workbook,
        sheet = 1,
        style = createStyle(
          fgFill = "#7dafff",
          numFmt = "comma"
        ),
        rows = 6:10,
        cols = 1:4,
        gridExpand = TRUE
      )
      
      saveWorkbook(my_workbook, file)
    }
  )

  
}

shinyApp(ui, server)

Continue Reading…

Collapse

Read More

A new study finds that partisanship matters more than the influence of the wealthy

Continue Reading…

Collapse

Read More

Reinforcement Learning: A Survey

Getting started with AI? Start here!

What AI-Driven Decision Making Looks Like

Optimization Problem in Deep Neural Networks

LIME: Explaining predictions of machine learning models (1/2)

P-values Explained By Data Scientist

Implementing and Analyzing different Activation Functions and Weight Initialization Methods Using Python

Natural Language Processing is Fun!

Conversational AI ? but where is the I?

Top 5 Mistakes of Greenhorn Data Scientists

Unity Machine Learning Agents Toolkit

The Best Refactoring You’ve Never Heard Of

How to Get Started with NLP – 6 Unique Methods to Perform Tokenization

Popular Machine Learning Applications and Use Cases in our Daily Life

Lorenz ’96 is too easy! Machine learning research needs a more realistic toy model.

How Deepfakes and Other Reality-Distorting AI Can Actually Help Us

Continue Reading…

Collapse

Read More

Article: The Scariest Thing About DeepNude Wasn’t the Software

Paper: The Elusive Model of Technology, Media, Social Development, and Financial Sustainability

Paper: Ethical Underpinnings in the Design and Management of ICT Projects

Paper: Mediation Challenges and Socio-Technical Gaps for Explainable Deep Learning Applications

Paper: Canada Protocol: an ethical checklist for the use of Artificial Intelligence in Suicide Prevention and Mental Health

Paper: Fairness and Diversity in the Recommendation and Ranking of Participatory Media Content

Paper: Global AI Ethics: A Review of the Social Impacts and Ethical Implications of Artificial Intelligence

Paper: A Study on the Prevalence of Human Values in Software Engineering Publications, 2015-2018

Continue Reading…

Collapse

Read More

Paper: Audits as Evidence: Experiments, Ensembles, and Enforcement

Paper: The Design of Mutual Information

Paper: Defining mediation effects for multiple mediators using the concept of the target randomized trial

Paper: Explaining Classifiers with Causal Concept Effect (CaCE)

Paper: Unbiased Learning to Rank: Counterfactual and Online Approaches

Paper: Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics

Continue Reading…

Collapse

Read More

Global Envelopes (GET)
Implementation of global envelopes with intrinsic graphical interpretation which can be used for graphical Monte Carlo and permutation tests where the …

Interactive Document for Working with Variance Analysis (VTShiny)
An interactive document on the topic of variance analysis using ‘rmarkdown’ and ‘shiny’ packages. Runtime examples are provided in the package function …

Multistage Allocation (R2BEAT)
Multivariate optimal allocation for different domains in one and two stages stratified sample design. R2BEAT extends the Neyman (1934) <doi:10.2307/ …

Collinearity Detection in a Multiple Linear Regression Model (multiColl)
The detection of worrying approximate collinearity in a multiple linear regression model is a problem addressed in all existing statistical packages. H …

Continue Reading…

Collapse

Read More

In the post (https://statcompute.wordpress.com/2019/07/14/yet-another-r-package-for-general-regression-neural-network), several advantages of General Regression Neural Network (GRNN) have been discussed. However, as pointed out by Specht, a major weakness of GRNN is the high computational cost required for a GRNN to generate predicted values based on a new input matrix due to its unique network structure, e.g. the number of neurons equal to the number of training samples.

For practical purposes, there is however no need to assign a neuron to each training sample, given the data duplication in real-world model development samples. Instead, a weighting scheme can be employed to reflect the frequency count of each unique training sample. A major benefit of the weight assignment is the ability to improve the efficiency of calculating predicted values, which depends on the extent of data duplicates. More attractively, the weighting application can bring up the possibility of using clustering or binning techniques to preprocess the training data so as to overcome the aforementioned weakness to a large degree.

Below is a demonstration showing the efficiency gain by using the weighting scheme in GRNN.

1. First of all, I constructed a sample data with duplicates to double the size of the original Boston dataset. Based on the constructed data, a GRNN named “N1” was trained.
2. Secondly, I generated another sample data by aggregating the above constructed data based on unique samples and calculating the weight of each unique data point based on its frequency. Based on the aggregated data, another GRNN named “N2” was also trained.

As shown in the output, predicted vectors from both “N1” and “N2” are identical. However, the computing time can be reduced to half by applying the weighting. All R functions used in the example can be found in https://github.com/statcompute/GRnnet/blob/master/code/grnnet.R.

For people interested in the SAS implementation of GRNN, two SAS macros are also available in https://github.com/statcompute/GRnnet/blob/master/code/grnn_learn.SAS and https://github.com/statcompute/GRnnet/blob/master/code/grnn_pred.SAS.

Continue Reading…

Collapse

Read More

Continue Reading…

Collapse

Read More

Finding neural network topologies is a problem with a rich history in evolutionary computing, or neuroevolution. This post will revisit some of the key ideas and outgoing research paths. Code related to this post is found here: [code link].

NEAT

In their 2002 paper, Kenneth Stanley & Risto Miikkulainen proposed the foundational algorithm NeuroEvolution of Augmenting Topologies (NEAT). I’ll focus on this algorithm as a starting point; for earlier developments please see Section 5.2 of this great review, Schaffer’s 1992 review, and Yao’s 1999 review. The NEAT paper introduces the ideas clearly and there are other great NEAT overviews, so to change it up I will try to present the algorithm with generic notation, which is perhaps useful for thinking about how to modify the algorithm or apply it to a new problem setting.

I’ve also made an implementation [code link] contained in a single python file; you might find it useful to see the entire algorithm in one place, or as a comparison if you also implement NEAT as an exercise. For a more robust implementation, see NEAT-Python (which the code is based on) and its extension PyTorch-NEAT.

Problem

NEAT addresses the problem of finding a computation graph . Each node has a bias, activation, and aggregation function, written , and each edge has a source and destination, a weight, and may be active or inactive, written  .

Searching through the space of these graphs amounts to searching through a space of neural networks. NEAT conducts this search using a few generic neuroevolution concepts, which I’ll focus on below, and often implements them with design decisions that can be relaxed or modified for different problems.

High-Level Method

NEAT iteratively produces a set of candidates , using a candidate partitioning where and a given function which measures a candidate’s quality. The candidates, partitions, and quality function are known as ‘population’, ‘species’, and ‘fitness’, respectively.

The candidate set (“population”, rectangle) contains partitions (“species”, circles), each containing candidate graphs (diamonds).

Each NEAT iteration returns a new candidate set and new partitioning, denoted as . Intuitively E is an ‘evolution step’ that produces a new ‘generation’. NEAT’s goal is to eventually output a ‘good’ candidate set . Typically good means that the best candidate has quality exceeding a goal threshold, . We then use this high-performing neural network on a task.

Evolution Steps

Each evolution step produces a new population using four ideas: mutation, crossover, fitness ranking, and partitioning.

Mutation randomly perturbs a candidate graph. In NEAT, mutations consist of adding or deleting a node, adding or deleting an edge, or perturbing a node or edge property (such as an edge’s weight or a node’s activation). Each mutation type occurs with a pre-specified probability, and involves a random perturbation; for instance an add-edge randomly chooses an edge location and weights are adjusted with Gaussian noise. One can design other mutations, such as resetting a weight to a new value.

An add-node mutation followed by an add-edge mutation. The add-node mutation splits an existing edge into two edges.

Crossover produces a new candidate by swapping properties of two existing candidates. In NEAT, roughly speaking if and have a matching node , then receives one of them randomly (similarly for edges). simply inherits non-matching nodes or edges. The notion of ‘matching’ is tricky due to isomorphic graph structures, so NEAT assigns an ID to each new node and edge, then uses these IDs for comparison (see 2.2 and 3.2 of the NEAT paper for details).  In part due to the added complexity, some papers leave out crossover completely.

NEAT crossover mechanism (diagram from the NEAT paper)

Fitness Ranking follows its name, first ranking candidates according to fitness, where means . Only the top (e.g. 20%) candidates are used for crossover and mutation. This locally biases the search towards candidates with high relative fitness.

Partitioning, or speciation, groups candidates according to a distance function . One use of the partitions is to promote diversity in the solution space by modifying each candidate’s fitness. To do so, NEAT defines a distance function and adjusts each candidate’s fitness based on its partition size. Each partition is guaranteed a certain number of candidates in the next generation based on the adjusted fitnesses.

Intuitively, a small partition contains graphs with relatively unique characteristics which might ultimately be useful in a final solution, even if they do not yield immediate fitness. To avoid erasing these characteristics from the search during fitness ranking, the small partition candidates receive guaranteed spots in the next phase.

Novel structures (diamonds, triangles) may ultimately yield a performance gain after further development, despite initially having lower fitness (light green) compared to common, developed structures with high fitness (dark green).

We can write this step as . We might alternatively view this step as just fitness re-ranking, , without requiring actual partitions, though without partitions it may be tricky to achieve the exact ‘guaranteed spots’ behavior.

The partitions could also be useful in problems requiring a collection of solutions rather than a single optimal solution. For instance, rather than just selecting the highest performing candidate, we might consider the best candidate in each partition as the final output of NEAT, thus producing a collection of networks, each maximizing fitness in a different way than the others (assuming a partitioning scheme that promotes diverse solutions).

Example Results

Let’s use the implementation [code link] to solve an xor problem and the Cartpole and Lunar-Lander gym environments.

To solve xor, NEAT finds a network with a single hidden node:

An xor network, including input (green), hidden (blue), and output (red) nodes. Labels show edge weights and node activations.

CartPole-v0 is easy to solve (even random search is sufficient), and NEAT finds a simple network without hidden units (for fun we’ll also construct an artificially complicated solution in the Variations section below):

A CartPole-v0 network.

LunarLander-v2 is more difficult, and NEAT finds a network with non-trivial structure:

A LunarLander-v2 network.

On the xor environment, NEAT creates around 10 partitions, on Cartpole just 1, and on LunarLander it tends to create 2-3 partitions. On these simple environments NEAT also performs similarly without crossover.

Variations As mentioned before, we may want NEAT to produce a diverse set of solutions rather than a single solution. To manually demonstrate this intuition, suppose I want NEAT to find a network that uses sigmoid activations, and one that uses tanh. To do so, I increased the activation parameter in the node distance function (the used in partitioning), then chose the highest scoring network from each partition. On Cartpole, the partitions now naturally separate into sigmoid and tanh networks:

While Cartpole is evidently simple enough for a network with no hidden layers, perhaps we want to follow a trend of using large networks even for easy problems. We can modify the fitness function to ‘reject’ networks without a certain number of connections, and NEAT will yield more complicated solutions:

A more complicated way to play Cartpole.

In particular, I added -1000 to the fitness when the network had less than k connections, starting with k=5 and incrementing k each time a candidate achieved max fitness at the current k (stopping at k=20).

Discussion & Extensions

Vanilla NEAT attempts to find both a network structure and the corresponding weights from scratch. This approach is very flexible and involves minimal assumptions, but could limit NEAT to problems requiring small networks. However, the key idea can still be applied or modified in creative ways.

Minimal Assumptions

NEAT represents an extreme on the spectrum of learned versus hand-crafted architectural biases, by placing few assumptions on graph structure or learning algorithm. At a very speculative level, such flexibility may be useful for networks with backward or long-range connections that may be difficult to hand design, or as part of a learning process which involves removing or adding connections rather than optimizing weights of a fixed architecture.

A more concrete example is the recent Weight Agnostic Neural Networks paper (Gaier & Ha 2019), where the authors aimed to find a model for a task by finding good network structures, rather than finding good weights for a fixed network structure; they use a single shared weight value in each network and evaluate fitness on multiple rollouts, with a randomly selected weight value for each rollout. In this case, a NEAT variant allowed finding exotic network structures from scratch, without requiring prior knowledge such as hand-designed layer types.

As a rough approximation, I modified the NEAT implementation so that each network only has a single shared weight value, and included more activation functions (sin, cos, arctan, abs, floor). Each run of evaluation sets the network’s shared weight to a randomly sampled value ( excluding ), and the network’s overall fitness is the average fitness over 10 runs. On XOR, NEAT finds a network with similar structure as before:

XOR network with a random shared weight value

This was just an initial experiment to give intuition, so check out the WANN paper for a good way of doing this for non-trivial tasks.

Scalability

One could also consider improving NEAT’s scalability. A high level strategy is to reduce the search space by restricting the search to topologies, searching at a higher abstraction level, or introducing hierarchy.

An example is DeepNEAT (Miikkulainen et al 2017), which evolves graph structures using NEAT, but with nodes representing layers rather than single neurons, and edges specifying layer connectivity. Weight and bias values are learned with back-propagation. The authors further extend DeepNEAT to CoDeepNEAT, which represents graphs with a two level hierarchy defined by a blueprint specifying connectivity of modules. Separate blueprint and module populations are evolved, with the full graph (module + blueprint) assembled for fitness evaluation.

Blueprint and Module populations. Each node in a Blueprint (hexagon) is a Module.

This view is quite general, allowing learning the internal structure of reusable modules as well as how they are composed. In the experiments the authors begin with modules involving known components such as convolutional layers or LSTM cells and evolve only specific parts (e.g. connections between LSTM layers), but one might imagine searching for completely novel, reusable modules.

Indirect Encodings

NEAT essentially writes down a description, or direct encoding, of every node and edge and their properties, then evolves these descriptions. The description size grows as the network grows, making the search space prohibitively large.

An alternative is to use a function to describe a network. For instance, we can evaluate a function at pairs of points from a grid to obtain a weighted adjacency matrix. This function is an example of an indirect encoding of the graph. Assuming the description of is small, we can describe very large networks by evaluating a suitable using a large grid or coordinate pattern. A neural network with a variety of activations that is evaluated in this manner is called a compositional pattern producing network (CPPN) [see, also].

HyperNEAT (Stanley et al. 2009) uses this idea to find network weights by evolving an indirect encoding function. HyperNEAT uses NEAT to evolve a (small) CPPN to act as , then evaluates at coordinates from a hyper-cube, resulting in weights of a (larger) network used for fitness evaluation.

Several works have adopted or extended ideas from HyperNEAT for a deep learning setting. Fernando et al. 2016 proposed the Differentiable Pattern Producing Network (DPPN) which evolves the structure of a weight-generating CPPN while using back-propagation for its weights. The authors evolve a 200 parameter that generates weights for a fully connected auto-encoder with ~150,000 weights, though it is for a small-scale MNIST image de-noising task. Interestingly the weight generating function learns to produce convolution-esque filters embedded in the fully connected network.

From [Fernando et al 2016]

From [Ha et al. 2016]

Wrapping Up

In this post we revisited a core technique for generating neural network topologies, and briefly traced some of its outgoing research paths. We took a brief step back from the constraints of pre-defined-layer architectures and searched through a space of very general (albeit small-scale) topologies. It was interesting to see how this generality has been refined towards some larger scale tasks, but also revisited.  We briefly saw how fitness re-ranking and partitioning can be used to yield a set of distinct solutions, which connects to other concepts that I may discuss further in future posts.

[Code]

Further Reading

• Designing Neural Networks through Neuroevolution
• On the automated, evolutionary design of neural networks: past, present, and future

Continue Reading…

Collapse

Read More

A new release 0.3.2 of the RPushbullet package is now on CRAN. RPushbullet is interfacing the neat Pushbullet service for inter-device messaging, communication, and more. It lets you easily send alerts like the one to the left to your browser, phone, tablet, … – or all at once.

This is the first new release in almost 2 1/2 years, and it once again benefits greatly from contributed pull requests by Colin (twice !) and Chan-Yub – see below for details.

Changes in version 0.3.2 (2019-07-21)

• The Travis setup was robustified with respect to the token needed to run tests (Dirk in #48)

• The configuration file is now readable only by the user (Colin Gillespie in #50)

• At startup initialization is now more consistent (Colin Gillespie in #53 fixing #52)

• A new function to fetch prior posts was added (Chanyub Park in #54). `

Courtesy of CRANberries, there is also a diffstat report for this release. More details about the package are at the RPushbullet webpage and the RPushbullet GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Continue Reading…

Collapse

Read More

David Allison points us to these two letters:

Alternating Assignment was Incorrectly Labeled as Randomization, by Bridget Hannon, J. Michael Oakes, and David Allison, in the Journal of Alzheimer’s Disease.

Change in study randomization allocation needs to be included in statistical analysis: comment on ‘Randomized controlled trial of weight loss versus usual care on telomere length in women with breast cancer: the lifestyle, exercise, and nutrition (LEAN) study,’ by Stephanie Dickinson, Lilian Golzarri-Arroyo, Andrew Brown, Bryan McComb, Chanaka Kahathuduwa, and David Allison, in Breast Cancer Research and Treatment.

It can be surprisingly difficult for researchers to simply say exactly what they did. Part of this might be a desire to get credit for design features such as random assignment that were too difficult to actually implement; part of it could be sloppiness/laziness; but part of it could just be that, when you write, it’s so easy to drift into conventional patterns. Designs are supposed to be random assignment, so you label them as random assignment, even if they’re not. The above examples are nothing like pizzagate, but it’s part of the larger problem that the scientific literature can’t be trusted. It’s not just that you can’t trust the conclusions; it’s also that papers make claims that can’t possibly be supported by the data in them, and that papers don’t state what the researchers actually did.

As always, I’m not saying these researchers are bad people. Honesty and transparency are not enuf. If you’re a scientist, and you write up your study, and you don’t describe it accurately, we—the scientific community, the public, the consumers of your work—are screwed, even if you’re a wonderful, honorable person. You’ve introduced buggy software in the world, and the published corrections, if any, are likely to never catch up.

P.S. Hannon, Oakes, and Allison explain why it matters that the design described as a “randomized controlled trial” wasn’t actually that:

By sequentially enrolling participants using alternating assignment, the researchers and enrolling physicians in this study were able to know to which group the next participant would be assigned, and there is no allocation concealment. . . .

The allocation method employed by Ito et al. allows the research team to determine in which group a participant would be assigned, and thus could (unintentionally) manipulate the enrollment. . . .

Alternating assignment, or similarly using patient chart numbers, days of the week, date of birth, etc., are nonrandom methods of group allocation, and should not be used in place of randomly assigning participants . . .

There are a number of disciplines (i.e., public health, community interventions, etc.) which commonly employ nonrandomized intervention evaluation studies, and these can be conducted with rigor. It is crucial for researchers conducting these nonrandomized trials to report procedures accurately.

Continue Reading…

Collapse

Read More

• Towards Generation of Visual Attention Map for Source Code
• Automatic Repair and Type Binding of Undeclared Variables using Neural Networks
• Predicting Merge Conflicts in Collaborative Software Development
• Characterizing Developer Use of Automatically Generated Patches
• Patterns of Effort Contribution and Demand and User Classification based on Participation Patterns in NPM Ecosystem
• The Design of Mutual Information
• Metamorphic Testing of a Deep Learning based Forecaster
• A Divide-and-Conquer Approach towards Understanding Deep Networks
• Estimation and Feature Selection in Mixtures of Generalized Linear Experts Models
• Task Selection Policies for Multitask Learning
• Bayesian Synthesis of Probabilistic Programs for Automatic Data Modeling
• Measuring the Transferability of Adversarial Examples
• Exploring Deep Anomaly Detection Methods Based on Capsule Net
• A Simple Uniformly Valid Test for Inequalities
• Discriminative Active Learning
• What does it mean to understand a neural network?
• Sequential online prediction in the presence of outliers and change points: an instant temporal structure learning approach
• Dynamical Systems as Temporal Feature Spaces
• Comprehensive Process Drift Detection with Visual Analytics
• Quick, Stat!: A Statistical Analysis of the Quick, Draw! Dataset
• A Causal Bayesian Networks Viewpoint on Fairness
• Confidentiality and linked data
• A study on the Interpretability of Neural Retrieval Models using DeepSHAP
• A Recursive Algebraic Coloring Technique for Hardware-Efficient Symmetric Sparse Matrix-Vector Multiplication
• Markov chain Monte Carlo algorithms with sequential proposals
• The Age of Incorrect Information: A New Performance Metric for Status Updates
• Agglomerative Attention
• Pathways to Good Healthcare Services and Patient Satisfaction: An Evolutionary Game Theoretical Approach
• Parallelism Theorem and Derived Rules for Parallel Coherent Transformations
• Low Power Receiver Front Ends: Scaling Laws and Applications
• Spectrum Sensing and Resource Allocation for 5G Heterogeneous Cloud Radio Access Networks
• Cover and variable degeneracy
• Variable degeneracy on toroidal graphs
• Tensor Methods for Finding Approximate Stationary Points of Convex Functions
• Energy-Efficient Activation and Uplink Transmission for Cellular IoT
• Smile, be Happy Emoji Embedding for Visual Sentiment Analysis
• Modeling the Uncertainty in Electronic Health Records: a Bayesian Deep Learning Approach
• On Rado conditions for nonlinear Diophantine equations
• Compressed Subspace Learning Based on Canonical Angle Preserving Property
• FoodX-251: A Dataset for Fine-grained Food Classification
• Microsoft Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine Translation
• On Happy Colorings, Cuts, and Structural Parameterizations
• The FAST Algorithm for Submodular Maximization
• Fast Graph Sampling Set Selection Using Gershgorin Disc Alignment
• Hybrid Offline-Online Design for UAV-Enabled Data Harvesting in Probabilistic LoS Channel
• Metadata Extraction from Raw Astroparticle Data of TAIGA Experiment
• On decomposing complete tripartite graphs into 5-cycles
• Emergency DC Power Support Strategy Based on Coordinated Droop Control in Multi-Infeed HVDC System
• A GPU implementation of the Discontinuous Galerkin method for simulation of diffusion in brain tissue
• On improving learning capability of ELM and an application to brain-computer interface
• On the Role of Time in Learning
• A local maximizer for lattice width of $3$-dimensional hollow bodies
• Necessary and sufficient condition for equilibrium of the Hotelling model
• Wong-Zakai approximations with convergence rate for stochastic partial differential equations
• Unsupervised Automatic Building Extraction Using Active Contour Model on Unregistered Optical Imagery and Airborne LiDAR Data
• A Note on M-convex Functions on Jump Systems
• Simple Automatic Post-editing for Arabic-Japanese Machine Translation
• Guaranteeing E2E QoS via Joint Radio and NFV Resource Allocation for 5G and Beyond
• An efficient method to construct self-dual cyclic codes of length $p^s$ over $\mathbb{F}_{p^m}+u\mathbb{F}_{p^m}$
• A fast direct solver for two dimensional quasi-periodic multilayered medium scattering problems
• A Simple BERT-Based Approach for Lexical Simplification
• Designing Unimodular Sequences with Optimized Auto/cross-correlation Properties via Consensus-ADMM/PDMM Approaches
• Multi-Level Order-Flow Imbalance in a Limit Order Book
• Avoiding Membrane Locking with Regge Interpolation
• Pointwise adaptive kernel density estimation under local approximate differential privacy
• Combinatorial t-designs from quadratic functions
• Markov-switching State Space Models for Uncovering Musical Interpretation
• A new bound for the crossing number of wrapped butterflies
• On the Global Convergence of Actor-Critic: A Case for Linear Quadratic Regulator with Ergodic Cost
• State Estimation in Visual Inertial Autonomous Helicopter Landing Using Optimisation on Manifold
• Delivery, consistency, and determinism: rethinking guarantees in distributed stream processing
• Wave solutions of Gilson-Pickering equation
• On the Equivalence of Youla, System-level and Input-output Parameterizations
• More Supervision, Less Computation: Statistical-Computational Tradeoffs in Weakly Supervised Learning
• Feature space transformations and model selection to improve the performance of classifiers
• Counterfactual Reasoning for Fair Clinical Risk Prediction
• Solving Mean-Payoff Games via Quasi Dominions
• Hybrid Model-Based and Data-Driven Wind Velocity Estimator for the Navigation System of a Robotic Airship
• Flux superperiods and periodicity transitions in quantum Hall interferometers
• ALFA: A Dataset for UAV Fault and Anomaly Detection
• An Artificial Spiking Quantum Neuron
• Metric Thickenings, Borsuk-Ulam Theorems, and Orbitopes
• Wall-crossings for Hassett descendant potentials
• Synchronization for KPZ
• Discourse Behavior of Older Adults Interacting With a Dialogue Agent Competent in Multiple Topics
• Autoencoding sensory substitution
• Learning Neural Networks with Adaptive Regularization
• Finite-Time Performance Bounds and Adaptive Learning Rate Selection for Two Time-Scale Reinforcement Learning
• TWEETQA: A Social Media Focused Question Answering Dataset
• An Approach Based on Bayesian Networks for Query Selectivity Estimation
• Perceptually Motivated Method for Image Inpainting Comparison
• Universal Non-Intrusive Load Monitoring (NILM) Using Filter Pipelines, Probabilistic Knapsack, and Labelled Partition Maps
• Compound TCP with Random Early Detection (RED): stability, bifurcation and performance analyses
• On the Evolution of U.S. Temperature Dynamics
• Tensor train-Karhunen-Loève expansion for continuous-indexed random fields using higher-order cumulant functions
• Resource theory of asymmetric distinguishability for quantum channels
• Splaying Preorders and Postorders
• New Paths from Splay to Dynamic Optimality
• Numerical study of vanishing and spreading dynamics of chemotaxis systems with logistic source and a free boundary
• Enabling Multi-Shell b-Value Generalizability of Data-Driven Diffusion Models with Deep SHORE
• Gradient Flow Based Discretized Kohn-Sham Density Functional Theory
• A Novel User Representation Paradigm for Making Personalized Candidate Retrieval
• FastV2C-HandNet: Fast Voxel to Coordinate Hand Pose Estimation with 3D Convolutional Neural Networks
• Multilevel Particle Filters for the Non-Linear Filtering Problem in Continuous Time
• Ranking sentences from product description & bullets for better search
• Myers-Briggs Personality Classification and Personality-Specific Language Generation Using Pre-trained Language Models
• Seedless Graph Matching via Tail of Degree Distribution for Correlated Erdos-Renyi Graphs
• Remarks on Gross’ technique for obtaining a conformal Skorohod embedding of planar Brownian motion
• An efficient estimator of the parameters of the Generalized Lambda Distribution
• A dynamic over games drives selfish agents to win-win outcomes
• Alternating Direction Method of Multipliers (ADMMs) Based Distributed Approach For Wide-Area Control
• Controlling Model Complexity in Probabilistic Model-Based Dynamic Optimization of Neural Network Structures
• Joint Language Identification of Code-Switching Speech using Attention based E2E Network
• A Constructive Proof of Jacobi’s Identity for the Sum of Two Squares
• Structural multiscale topology optimization with stress constraint for additive manufacturing
• Motorway Traffic Flow Prediction using Advanced Deep Learning
• Entanglement-assisted Quantum Codes from Algebraic Geometry Codes
• CA-RefineNet:A Dual Input WSI Image Segmentation Algorithm Based on Attention
• The Elusive Model of Technology, Media, Social Development, and Financial Sustainability
• Micro, Meso, Macro: the effect of triangles on communities in networks
• Linked partition ideals, directed graphs and $q$-multi-summations
• Multimodal deep networks for text and image-based document classification
• Mitigating the Hubness Problem for Zero-Shot Learning of 3D Objects
• Edge-bipancyclicity of bubble-sort star graphs
• Proper Orientation Number of Triangle-free Bridgeless Outerplanar Graphs
• Reliability-Latency Performance of Frameless ALOHA with and without Feedback
• GLOSS: Generative Latent Optimization of Sentence Representations
• Single-Component Privacy Guarantees in Helper Data Systems and Sparse Coding
• Sequence Level Semantics Aggregation for Video Object Detection
• A Dual Memory Structure for Efficient Use of Replay Memory in Deep Reinforcement Learning
• Energy cost for target control of complex networks
• Optimal Control of a Hot Plasma
• Improving the Harmony of the Composite Image by Spatial-Separated Attention Module
• Investigation on N-gram Approximated RNNLMs for Recognition of Morphologically Rich Speech
• Stabilized Barzilai-Borwein method
• To Model or to Intervene: A Comparison of Counterfactual and Online Learning to Rank from User Interactions
• Concept-Centric Visual Turing Tests for Method Validation
• One-dimensional minimizers for a diffuse interface generalized antiferromagnetic model in general dimension
• Stochastic Galerkin finite volume shallow flow model: well-balanced treatment over uncertain topography
• On the Polarization of Rényi Entropy
• Flag-transitive non-symmetric $2$-designs with $(r,λ)=1$ and exceptional groups of Lie type
• Multi-hop Federated Private Data Augmentation with Sample Compression
• A Neural Turing~Machine for Conditional Transition Graph Modeling
• Noise-Stable Rigid Graphs for Euclidean Embedding
• Empirical Coordination Subject to a Fidelity Criterion
• Emergence of a bicritical end point in the random crystal field Blume-Capel model
• Shadow Simulated Annealing algorithm: a new tool for global optimisation and statistical inference
• RaKUn: Rank-based Keyword extraction via Unsupervised learning and Meta vertex aggregation
• Exponential decay of correlations in the 2D random field Ising model
• Improved Penalty Algorithm for Mixed Integer PDE Constrained Optimization (MIPDECO) Problems
• Anonymous and confidential file sharing over untrusted clouds
• Out-of-core singular value decomposition
• Neural network regression for Bermudan option pricing
• Fast Algorithms and Theory for High-Dimensional Bayesian Varying Coefficient Models
• Proximal Policy Optimization with Mixed Distributed Training
• Unsupervised Fault Detection in Varying Operating Conditions
• Color Cerberus
• Naver Labs Europe’s Systems for the WMT19 Machine Translation Robustness Task
• DeepSUM: Deep neural network for Super-resolution of Unregistered Multitemporal images
• Robust Nonlinear Component Estimation with Tikhonov Regularization
• An Efficient Framework for Visible-Infrared Cross Modality Person Re-Identification
• Labels instead of coefficients: a label bracket which dominates the Jones polynomial, the Kuperberg bracket, and the normalised arrow polynomial
• Detecting and Simulating Artifacts in GAN Fake Images
• A new combinatorial interpretation of the Fibonacci numbers squared
• Tracking sex: The implications of widespread sexual data leakage and tracking on porn websites
• Inapproximability within W[1]: the case of Steiner Orientation
• Federated Reinforcement Distillation with Proxy Experience Memory
• Computing the Kreiss Constant of a Matrix
• Deep Sequential Mosaicking of Fetoscopic Videos
• Subgroups of simple primitive permutation groups defined by unordered relations
• A Dimension-free Algorithm for Contextual Continuum-armed Bandits
• Dynamic Tube MPC for Nonlinear Systems
• Asking Clarifying Questions in Open-Domain Information-Seeking Conversations
• Should we Embed? A Study on the Online Performance of Utilizing Embeddings for Real-Time Job Recommendations
• Achievable Data Rate for URLLC-Enabled UAV Systems with 3-D Channel Model
• Addressing Delayed Feedback for Continuous Training with Neural Networks in CTR prediction
• The Elicitation of Prior Distributions for Bayesian Responsive Survey Design: Historical Data Analysis vs. Literature Review
• The Many AI Challenges of Hearthstone
• Recovery Guarantees for Compressible Signals with Adversarial Noise
• Improved Hybrid Layered Image Compression using Deep Learning and Traditional Codecs
• Posterior Predictive Treatment Assignment Methods for Causal Inference in the Context of Time-Varying Treatments
• On Cyclic Finite-State Approximation of Data-Driven Systems
• Automated Playtesting of Matching Tile Games
• Efficient Video Generation on Complex Datasets
• Improved Budgeted Connected Domination and Budgeted Edge-Vertex Domination
• Probability inequalities for high dimensional time series under a triangular array framework
• Experimental machine learning quantum homodyne tomography
• Forced synchronization of an oscillator with a line of equilibria
• A comparison of European and Asian options under Markov additive processes
• Medical Concept Representation Learning from Claims Data and Application to Health Plan Payment Risk Adjustment
• Hotelling Games with Multiple Line Faults
• Bayesian Wavelet Shrinkage with Beta Priors
• The saturation assumption yields optimal convergence of two-level adaptive BEM
• Revealing posturographic features associated with the risk of falling in patients with Parkinsonian syndromes via machine learning
• Facebook FAIR’s WMT19 News Translation Task Submission
• Audits as Evidence: Experiments, Ensembles, and Enforcement
• Zero-sum subsequences in bounded-sum $\{-r,s\}$-sequences
• Multi-scale Graph-based Grading for Alzheimer’s Disease Prediction
• Batch-Shaped Channel Gated Networks
• A Geometric Perspective on Quantum Parameter Estimation

Continue Reading…

Collapse

Read More

The Design of Mutual Information

We derive the functional form of mutual information (MI) from a set of design criteria and a principle of maximal sufficiency. The (MI) between two sets of propositions is a global quantifier of correlations and is implemented as a tool for ranking joint probability distributions with respect to said correlations. The derivation parallels the derivations of relative entropy with an emphasis on the behavior of independent variables. By constraining the functional according to special cases, we arrive at its general functional form and hence establish a clear meaning behind its definition. We also discuss the notion of sufficiency and offer a new definition which broadens its applicability.

Metamorphic Testing of a Deep Learning based Forecaster

A Divide-and-Conquer Approach towards Understanding Deep Networks

Estimation and Feature Selection in Mixtures of Generalized Linear Experts Models

Task Selection Policies for Multitask Learning

Bayesian Synthesis of Probabilistic Programs for Automatic Data Modeling

Measuring the Transferability of Adversarial Examples

Exploring Deep Anomaly Detection Methods Based on Capsule Net

A Simple Uniformly Valid Test for Inequalities

Discriminative Active Learning

What does it mean to understand a neural network?

Sequential online prediction in the presence of outliers and change points: an instant temporal structure learning approach

Dynamical Systems as Temporal Feature Spaces

Comprehensive Process Drift Detection with Visual Analytics

Quick, Stat!: A Statistical Analysis of the Quick, Draw! Dataset

A Causal Bayesian Networks Viewpoint on Fairness

Confidentiality and linked data

A study on the Interpretability of Neural Retrieval Models using DeepSHAP

A Recursive Algebraic Coloring Technique for Hardware-Efficient Symmetric Sparse Matrix-Vector Multiplication

Markov chain Monte Carlo algorithms with sequential proposals

The Age of Incorrect Information: A New Performance Metric for Status Updates

Agglomerative Attention

Continue Reading…

Collapse

Read More

We at Win-Vector LLC have some big news.

We are finally porting a streamlined version of our R vtreat variable preparation package to Python.

vtreat is a great system for preparing messy data for suprevised machine learning.

The new implementation is based on Pandas, and we are experimenting with pushing the sklearn.pipeline.Pipeline APIs to their limit. In particular we have found the .fit_transform() pattern is a great way to express building up a cross-frame to avoid nested model bias (in this case .fit_transform() != .fit().transform()). There is a bit of difference in how object oriented APIs compose versus how functional APIs compose. We are making an effort to research how to make this an advantage, and not a liability.

The new repository is here and the first example regression is here. Next up is classification (likely natively multinomial this time). After that a few validation suites to prove the two vtreat systems work similarly. And then we have some exciting new capabilities.

The first application is going to be a shortening and streamlining of our current 4 day data science in Python course (while allowing more concrete examples!).

This also means data scientists who use both R and Python will have a few more tools that present similarly in each language.

Here is a non-trivial classification example.

Continue Reading…

Collapse

Read More

Send Formatted Messages, Images and Objects to Microsoft ‘Teams’ (teamr)
Package of wrapper functions using R6 class to send requests to Microsoft ‘Teams’ <

Normalized Power Prior Bayesian Analysis (NPP)
Posterior sampling in several commonly used distributions using normalized power prior as described in Duan, Ye and Smith (2006) <doi:10.1002/env.75 …

Diagonally Dominant Principal Component Analysis (ddpca)
Consider the problem of decomposing a large covariance matrix into a low rank matrix plus a diagonally dominant matrix. This problem is called Diagonal …

Wrappers for ‘GDAL’ Utilities Executables (gdalUtilities)
R’s ‘sf’ package ships with self-contained ‘GDAL’ executables, including a bare bones interface to several of the ‘GDAL’-related utility programs colle …

Continue Reading…

Collapse

Read More

Generative adversarial networks (GAN) are a powerful subclass of generative models. Despite a very rich research activity leading to numerous interesting GAN algorithms, it is still very hard to assess which algorithm(s) perform better than others. We conduct a neutral, multi-faceted large-scale empirical study on state-of-the art models and evaluation measures. We find that most models can reach similar scores with enough hyperparameter optimization and random restarts. This suggests that improvements can arise from a higher computational budget and tuning more than fundamental algorithmic changes. To overcome some limitations of the current metrics, we also propose several data sets on which precision and recall can be computed. Our experimental results suggest that future GAN research should be based on more systematic and objective evaluation procedures. Finally, we did not find evidence that any of the tested algorithms consistently outperforms the original one. Are GANs Created Equal A Large-Scale Study

Continue Reading…

Collapse

Read More

Category

• Advanced Modeling

Tags

• Linear Regression
• Logistic Regression
• R Programming

Regression analysis is one of the most demanding machine learning methods in 2019. One group of regression analysis for measuring effects and to evaluate a policy program is Difference-in-Difference. This method is well suited for benchmarking and finding improvements for optimization in organizations. It can, therefore, be used to design organizations so they generate more value for employees and customers. In this article, you learn how to do difference-in-difference in R.

Methodology

The difference in differences (DiD) method is a statistical technique or quasi-experimental design method, and it is used primarily in the social sciences and econometrics. In social science, it is sometimes called a “controlled before-and-after” study.

The DiD method involves comparing results from two groups, with data from each group being recorded over two time periods. One group (the control group) is not exposed to any treatment or intervention whatsoever; the other (treatment group) is exposed to a treatment or intervention before or during one of the two time periods. The same observations are made in both groups over each time period.

The data is analyzed by first calculating the difference in first and second time periods, and then subtracting the average gain (or difference) in the control group from the average gain (or difference) in the treatment group.

Difference-in-Difference Estimator

In order to make the DiD estimator lets consider the main assumption of DID:
Counterfactual Levels for treated and non-treated can be different, but their time variation is similar.

Let us explain this with statistics:
$$ E(Y0(t1) – Y0(t0) | D=1)= E( Y0(t1) – Y0(t0) | D=0) $$
$$ [E(Y0(t1)|D=1) is counterf] $$

In the absence of treatment, change in treated outcome would have been as a change in non-treated outcome, i.e. changes in the economy or life-cycle, etc (unrelated to treatment) affect the two groups in a similar way.

This implies an relaxing assumption:
$$ E(Y0 | X, D=0) = E( Y0 | X,D=1) $$
$$ E( Y1 | X,D=0)= E( Y1 | X,D=1) $$

Selectivity bias is allowed even conditional on X, but only through an individual fixed effect (i.e. time constant).

Let us again consider the main assumption of DID:
$$ E(Y0(t1) – Y0(t0) | D=1) = E( Y0(t1) – Y0(t0) | D=0) $$

It is possible to show how this assumption is used to generate a “control group” that can be substituted in for the missing counterfactual:

$$
TTE = E( Y1(t1) – Y0(t1) | D=1)
$$
$$
= E( Y1(t1) – Y0(t0) + Y0(t0) -Y0(t1) | D=1)
$$
$$
= E( Y1(t1) – Y0(t0) | D=1) – E( Y0(t1) – Y0(t0) | D=1) [2. term unobs]
$$
$$
= E( Y1(t1) – Y0(t0) | D=1) – E( Y0(t1) – Y0(t0) | D=0) [2. term obs]
$$

Formally let us show that this can be written in a regression framework with individual fixed effects and time fixed effects:
$$ Y(it) = a(t)+b*D(it)+m(i)+u(it) $$
Where a(t)=time effect, m(i)=individual effect

Then we can write the model as:
$$ Y(i1) -Y(i0) = [a(1)-a(0)]+ b*[D(i1)-D(i0)] +[u(i1)-u(i0)] $$

And the fixed-effect estimator reduces to:
$$ b = E [Y(i1)-Y(i0)|D=1] – E [Y(i1)-Y(i0)|D=0] $$

Sample version of this is the DiD estimator, where assumption in this model is:
$$ E [u(i1)-u(i0)|D=1] = E [u(i1)-u(i0)|D=0] $$

Methodological Assumptions for Difference in Differences

When doing a DiD method analysis we assume that the composition of the groups being studied are stable over the time period we are concerned about. We also assume there are no spillover effects, the amount of treatment or intervention given is not determined by the outcome, and that both groups being studied have parallel trends in their outcome—i.e., if no treatment was given, the difference between the data from the two groups would have a consistent difference over time.

The most important assumption in DiD is the parallel trends assumption. It is, therefore, necessary to be cautious about studies that do not graphically show these trends. If there is no convincing graph that shows the parallel trends in the pre-treatment outcomes for the treatment and control groups, be cautious. If the parallel trends assumption holds and we can credibly rule out any other time-variant changes that may confound the treatment, then DiD is a trustworthy method.

The difference in difference method is intuitive and fairly flexible; it will show a causal effect from observational data if the basic assumptions are met. Since it focuses on change, rather than the absolute levels, the groups being compared can start at different levels. Another key strong point to the DiD method is that it accounts for change due to factors other than the treatment or intervention being studied.

Load dataset in R

First we will load the dataset in R and do descriptive statistics:

data_file = '../_data/did_data.dta'
if (!file.exists(data_file)) {
    download.file(url = 'https://drive.google.com/uc?authuser=0&id=0B0iAUHM7ljQ1cUZvRWxjUmpfVXM&export=download', 
                  destfile = data_file)
}

dataf = haven::read_dta(data_file)
skim(dataf, work, year, children) %>% 
    skimr::kable(digits = 0)
 Skim summary statistics    
n obs: 13746      
n variables: 11      
Variable type: numeric   
variable    missing    complete      n       mean       sd      p0     p25     p50     p75     p100      hist   
----------  ---------  ----------  -------  ---------  ------  ------  ------  ------  ------  ------  ----------
##  children       0        13746      13746     1.19      1.38     0       0       1       2       9      ▇▂▁▁▁▁▁▁ 
##    work         0        13746      13746     0.51      0.5      0       0       1       1       1      ▇▁▁▁▁▁▁▇ 
##    year         0        13746      13746    1993.35    1.7     1991    1992    1993    1995    1996    ▇▇▁▇▇▁▆▆

Now let us construct dichotom variables for (a) before-and-after the EITC takes effect in 1994 and (b) the treatment group (1 or more children). We’ll keep these as logicals:

dataf = dataf %>%
mutate(post93 = year >= 1994, anykids = children >= 1)

Difference-in-difference regression in R

The regression equation is:

work=β0+δ0post93+β1anykids+δ1(anykids×post93)+ε

The “difference-in-differences coefficient” is δ1, which indicates how the effect of kids changed after the EITC went into effect. Let’s take a second to plot this:

ggplot(dataf, aes(post93, work, color = anykids)) + geom_jitter() + theme_minimal()

The above coding gives us the following plot:

Let us also plot the variable means:

ggplot(dataf, aes(year, work, color = anykids)) +
    stat_summary(geom = 'line') +
    geom_vline(xintercept = 1994) +
    theme_minimal()

No summary function supplied, defaulting to `mean_se()

The above coding gives us the following plot:

The parallel trends assumption looks good. However, parallel trends are nonlinear.
Let us make the regression, which is created by using a linear probability model, where the DiD coefficient is the interaction coefficient δ1(anykids×post93):

model = lm(work ~ anykids*post93, data = dataf)
summary(model)

Call:
lm(formula = work ~ anykids * post93, data = dataf)
 
Residuals:
    Min      1Q  Median      3Q     Max 
-0.5755 -0.4908  0.4245  0.5092  0.5540 

Coefficients:
                        Estimate Std. Error t value Pr(>|t|)    
(Intercept)             0.575460   0.008845  65.060  < 2e-16 ***
anykidsTRUE            -0.129498   0.011676 -11.091  < 2e-16 ***
post93TRUE             -0.002074   0.012931  -0.160  0.87261    
anykidsTRUE:post93TRUE  0.046873   0.017158   2.732  0.00631 ** 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.4967 on 13742 degrees of freedom
Multiple R-squared:  0.0126,	Adjusted R-squared:  0.01238 
F-statistic: 58.45 on 3 and 13742 DF,  p-value: < 2.2e-16

The above model shows that the EITC increased work by 5% among families with at least 1 child. In the second plot, the blue line goes from about 45% prior to 1994 to about 50%, afterward, due to the 5% increase in the model.

• Find the best predictive model using R/caret package/modelgrid
• Variable Selection Methods: Lasso and Ridge Regression in Python
• Bootstrap with Monte Carlo Simulation in Python
• Monte Carlo Simulation and Statistical Probability Distributions in Python
• K-Nearest Neighbors (KNN) with Python

Continue Reading…

Collapse

Read More

The useR! Conference was held in Toulouse, France and for me this
was my second useR! after my first in Brisbane last year. This time
around I wanted to write about my experiences and some highlights
similar to my post on the RStudio::Conference 2019 & Tidyverse Dev
Day
earlier this year. This blog post will be divided into 4 sections: Programming, Shiny, {Packages}, and Touring Toulouse.

You can find slides and videos (in a week or so) in:

• useR Materials Github Repo (also contains workshop
  stuff) courtesy of Suthira Owlarn

• useR Official
  Website

• Timestamps for the Keynote Presentations courtesy of David
  Smith

As usual there were many talks that I didn’t get to go to as there are
around 3~5 tracks across different rooms featuring talks on a certain
aspect of R such as Shiny, Modelling, Data handling, DevOps, Education,
etc. In the coming weeks I’ll also add video links to
the presentations below when they become available from R Consortium’s
Youtube channel.

Let’s begin!

Programming

Enhancements to data tidying: Hadley Wickham

Acknowledging the difficulty of spread() and gather() you might have heard of the creation of the pivot_wider() and pivot_longer() functions in recent
months. You really should take a look at the work-in-progress
Vignette for a comprehensive understanding of the new functions but the
talk featured some live-coding by Hadley
(Code)
and some cool spread/gather animations via Charco Hui’s masters’
thesis.

For more material you might be interested in Hiroaki Yutani’s tidyr
1.0.0 presentation
from June’s Tokyo.R meetup. It’s mainly in Japanese but there are lots
of code and explanatory graphics that may aid you in visualizing how the
new functions work. You can also read a short English summary of the
talk here.

n() cool dplyr things: Romain Francois

• Slides

Taking the tidy data principles into account but for grouped data,
Romain Francois talked about the new group_*() functions in the
{dplyr} package.

While in previous versions of {dplyr} working in a tidy manner with
groups was done with group_by() then dplyr::do(), the latter
function has been deprecated and have been largely replaced by the
{purrr} family of functions instead. In this context the group_map(),
group_modify(), and group_walk() functions iterate like the {purrr}
functions but instead over groups. You can apply the functions you want
to apply to each group inline via a lambda, ~ (as below), or you can
specify a function directly without the lambda.

The group_split() operates similarly to base::split() but splits by
groups, the output being a list of sliced groups. The group_keys()
function returns you the exact grouping structure of the data you used
group_by() on, allowing you to check that the structure is right
before you start applying functions on your data. group_data() and
group_rows() gives you different kind of information about your
grouped data as can be seen below.

To shorten the group_by() %>% summarize() workflow you could instead
use the summarize_at() function. You can select specific columns with
vars(), then actions via a lambda, ~, and you can specify multiple
functions with list().

Romain also talked about the {dance} package which is mainly used to experiment and test out possible new {dplyr} functions by leveraging the
relatively new {vctrs} and {rlang} packages’ features. The package has a theme of using famous dance moves as the function names!

Reusing tidyverse code – Lionel Henry

• Slides

Lionel Henry talked about programming using {tidyverse} functions. As an
introduction he went over data masking in {dplyr} and how it is
optimized for interactive coding and single-use %>%s. The usage of
non-standard evaluation (NSE) makes analyses easy as you can focus on the data rather than the
data structure. However, we hit a stumbling block when it comes to when
we want to create custom functions to program with {dplyr}. This is the
difference between computing in the work space (as needed) versus
computing in a data mask.

This is where tidyeval comes into play via {rlang} for flexible and robust programming in the tidyverse. However {rlang} confused a lot of
people due to the strange new syntax it introduced such as the !!,
!!!, and enquo(). Also, it introduced new concepts such as
quasi-quotation and quosures that made it hard to learn for people
especially with those without a programming background. Acknowledging
this obstacle, was introduced to make creating tidyeval
functions easier. The new (read as “curly-curly”) operator was
inspired by the {glue} package and is a short cut for !!enquo(var).

Shiny

Keynote #2: Shiny apps and Reproducibility – Joe Cheng

• Slides

Compared to a R script or R Markdown document, reproducibility suffers
in Shiny apps as the outputs are transient and not archivable.
RStudio’s Joe Cheng talked about how reproducible analysis with Shiny is inconvenient as reenacting the
user’s interaction steps is necessary. A case for having a simple
CLICK button to view/download a reproducible artifact can be seen in
various industries such as:

• ex. Drug research/pharma validation (workflow)
• ex. Teaching: statistical concepts and code snippets
• ex. Gadgets/add-ins: building ggplots, regex, and SQL queries then
  insert the code into source/console editor

The different possible outputs we might want from a Shiny app are:

• To download the RMD or R file as the artifact
• To download a ZIP with source code & data, other supporting files,
  and the actual rendered result

From there Joe talks about how there are a number of options available
such as :

1. Copy-paste: Have a Shiny app and RMD report
   • Pros: Copy-pasted code is high fidelity and easy to understand
   • Cons: Two copies must be kept in sync and method will not work for
     more dynamic apps
2. Lexical analysis: automatically generate scripts from app source
   code (static analysis and heuristics)
   • Pros: Easy to add to app
   • Cons: Not ALL apps can be translated automatically
   • Generated code may not be camera ready as it may contain lots of
     code relating to the Shiny app’s structure
3. Programmatic: Meta-programming techniques to write code for dual
   purposes (execute interactive and export static)
   • Pros: Flexible
   • Cons: High learning curve and significant effort needed to adapt
     old Shiny apps

In light of the various pros and cons of the above options Joe with the
help of Carson Sievert created the…

{shinymeta} package

There are four main steps to follow when using {shinymeta}:

1. Identify the domain logic inside the code and separate it from
   Shiny’s reactive structure
   • Activate meta mode with withMetaMode() or expandChain()
   • Use metaReactive() to create a reactive() that returns a code
     expression
   • Other functions to return code include metaObserve(),
     metaRender(), etc.
   • You can also wrap the code you want with metaExpr() inside
     function

1. Within the domain logic you identified, identify references to
   reactive values and expressions that need to be replaced with static
   values and static code
   • De-reference reactive values with !!
   • Replace reactive values with the actual values

1. At run time, choose which pieces of domain logic to expose to
   the user
   • expandChain(): turns !! code into variable and introduces code
     snippet above the function
   • The chain of variable declarations grow upwards as you sequentially
     expand the meta-objects

1. Present the code to the user!
   • Use outputCodeButton() to add a button for a specific output
   • Use displayCodeModal() to display underlying code
   • Use downloadButton() to allow people to click and download a R
     script or RMD report
   • Use buildScriptBundle or buidlRmdBundle() to generate .zip
     bundles dynamically

Some of the limitations and future directions Joe, Carson, and the rest
of the Shiny team acknowledge are that:

• The formatting of the code can be improved (white
  space not preserved)
• Future compatibility with Shiny async
• So far {shinymeta} only covers reproducing “snapshots” of the app
  state
• More work and thinking needs to be done to reproduce a “notebook”
  style record of the how/why/what of the multiple iterations of
  interactive usage that was needed to get to a certain result and
  output

There’s a lot to take in (this was probably the toughest talk for me to
explain in this post…), so besides watching the keynote
talk yourself you
can also take a look at the shinymeta package
website.

{golem}: Shiny apps in production – Vincent Guyader

• Slides
• {golem}

Vincent Guyader, from another French R organization
ThinkR, talked about the new {golem}
package which creates a nice framework for building robust
production-ready Shiny apps.

One of the key principles in R is when you are repeatedly writing or
using the same code or functions then you should write a package, and
this is no different for Shiny apps as well. The reasons Vincent stated
were:

• Easy dependency, version, documentation management
• Easy installation and deployment

With the package infrastructure, you need to have the ui.R and
server.R (app_ui.R and app_server.R respectively in {golem}) in
the R directory and all you need to run your app is the run_app()
function.

{golem} also has functions that make it easy to deploy your app via R
Studio Connect, shinyproxy, Shiny server, heroku, etc.

For styling your app with customized JavaScript and CSS files you can
easily add them to your Shiny app package directory via the
add_js_file() and add_css_file() functions. You can do similar but
with modules with add_module(). As {golem} is a package you have all
the great attributes of an R package available to you such as unit
testing, documentation, and continuous integration/deployment!

Our journey with Shiny: Some packages to enhance your applications – Victor Perrier & Fanny Meyer

• Slides

Victor Perrier and Fanny Meyer from dreamRs talked about
the various Shiny packages that can extend the functionality of your
Shiny applications!

The first and probably the most well-known of this group is the
{shinyWidgets} package which gives you a variety of cool custom widgets that you can add to make your Shiny app via JavaScript and CSS.

Next, wondering about how exactly users interacted with their Shiny apps
and whether they used the included widgets the dreamRs team created the
{shinylogs} package. This packages records any and all inputs that are changed as well as the outputs and errors. This is done by storing the
JavaScript objects via the
localForage JavaScript
library. With this in place shiny developers can see the number of
connections per day, the user agent family, most viewed tabs, etc.

The {shinybusy} package gives a user feedback when a server operation running or busy such as a spinning circle, a moving bar, or even any
kind of gif you choose!

Last but not least is the {shinymanager} package which allows you to administrate and manage who can access your application and protects the source code of your app until authentication is successful!

The dreamRs organization are also the organization that created the
{esquisse} package that lets you interactively make ggplot2 graphs with an RStudio addin!

Talking about packages leads me to the next section…

Packages

Summary of developments in R’s data.table package – Arun Srinivasan

• Slides
• {data.table}

I’ve been curious about data.table so I decided to go to this talk
to learn more from Arun Srinivasan, one of the authors of the package. Starting off
with some trivia, I finally learned that the reason for the seal on the
hex sticker is because seals make an “aR! aR! aR!” sound according to
{data.table} creator Matt Dowle, which I thought was pretty great!

Compared to a year ago there has been a lot of change and progress in
{data.table}:

A key principle of {data.table} is that there are no dependencies or
imports in the package!

The general form of using {data.table} is as follows:

Arun also showed us some examples:

At the end he also talked about the new optimization and functionalities
in the package.

• for ‘i’: auto-indexing and parallel subsets (columns processed in
  parallel)
• for ‘j’: using GForce
• for ‘by’: parallelization of radix ordering
• new functionality: froll(), coalesce(), and nafill()

At the end of the talk Arun thanked the 69 people (among them Michael
Chirico, Philippe Chataignon, Jan Gorecki, etc.) who have contributed a
lot to what {data.table} is today!

{polite} – Dmytro Perepolkin

• Slides
• {polite}

The {polite} package is one I’ve been used for over a year now (you
might’ve seen me use it in my soccer or TV data viz) and I was delighted
to hear that the creator was giving a LT on it! Dmytro began with a few do’s and don’ts concerning user-agents and being explicit about them:

Secondly, you should always check the robots.txt for the website which is a file that
stipulates various conditions for scraping activity. This can be done
via Peter Meissner’s {robotstxt} package or by checking the output from polite::bow("theWebsiteYouAreScraping.com")(polite::bow() function
is what establishes the {polite} session)!

After getting permission you also need to limit the rate at which you
scrape, you don’t want to overload the servers of the website you are
using, so no parallelization! This can be done with the {ratelimitr}
package, purrr::slowly() while the {polite} package automatically
delays by 5 seconds when you run polite::scrape().

After scraping, you should definitely cache your responses with {memoise}, which is what is used inside the polite::scrape() function. Also, wrap your scraper function
with something like purrr:::safely() so it returns a list of two
components, a “result” for successes and “error” object for errors in
your scraping.

You can also read his blog post on the talk
here which explains a
bit more about the polite::use_manners() function that allows you to
include {polite} scrapers into your own R packages.

goodpractice: good pkg development Hannah Frick

• Slides
• {goodpractice}

Hannah Frick from Mango Solutions talked about {goodpractice}, a package that gives you advice about good practices for building an R package. By
running goodpractice::gp() it does static code analysis and can run
around ~200 of the checks available.

A cool thing you can do is that you can customize the different checks
it runs, set your own standards beforehand and run the checks based on
those standards with the make_check() and make_prep() functions.
It’s a great package that I’ve used before at work and for my own
packages so definitely try it out!

R Community

The development of {datos} package for the R4DS Spanish translation – Riva Quiroga

• Slides
• {datos}

Riva Quiroga talked about translating
the “R for Data Science” book and R data sets into Spanish. This came
about as a fact that learning R (or any programming language) can be
tough for a non-English speaker as it means you have to not only learn
the programming but figuring out what the documentation and use cases in
English even mean. To address this language gap the R4DS Spanish
translation community project was born, Ciencia de
Datos on Github! Through Github and
slack the organization sought to translate both the book and the various
data sets available in base R, for example: turning “diamonds” into “dimantes”.
However, they found that simply trying to rename() everything was not
sustainable so they had to find an alternative. This alternative ended
up being the {datalang}
package.

This package (created by RStudio’s Edgar Ruiz) uses a YAML spec file
translating to the language you want for the variable names, value
names, help files, etc. After creating the spec file you just have to
add it as an argument into the
datalang::translate_data()/translate_folder() function and you’ll have
a translated data set! The R para Ciencia de Datos Twitter also hosts a Spanish version of #TidyTuesday called #DatosDeMiercoles so check it
out!

Another thought I had after this presentation was that maybe this might
be a good idea for Japanese?

R Consortium Working Groups – Joseph Rickert

• Slides

RStudio’s Joe Rickert talked about R
Consortium’s Workings Groups which is an initiative to foster innovation
among individuals and companies. Any individual or a group can apply to
create a working group to explore what R and other technologies can do
in a certain field of interest. Throughout the talk Joe gave examples of
successful workings groups such as:

• Census Working Group
• R in Pharma (which now has its own
  conference!)
• R Validation Hub: enabling the usage of R in regulatory settings
• R Community Diversity and
  Inclusion
• And more!

As advice for potential working groups Joe said that one should pick a
project with a very wide scope which can benefit from collaboration
between members and which can benefit a large portion of the R
community.

Keynote: #3 ‘AI for Good’ in the R and Python ecosystems – Julien Cornebise

In the last keynote of the conference Julien Cornebise talked about using technology tools for good using lots of examples throughout his
life for both good and bad projects.

Here are some quotes I was able to jot down:

On using technology for good:

“Technology is not a solution it is an accelerator, essentially you just have a better optimizer, you’re just fitting better to the incentives we have around us a society.”

On the motivation of getting involved in #DataForGood projects:

“Are you here to solve the problem or are you here for a really cool application of your fantastic new theory and algorithm?”

On “hackathon syndrome” of many solutions to #DataForGood problems:

“Github is a big cemetary of really good ideas … where do we find software engineers, where do we find the designers, how do we go from the solution to the project to a real product that can be used by many many people?”

Some of the projects he talked about were:

• Decode
  Darfur:
  Identifying remote burnt/destroyed villages in the Darfur region to
  provide credible evidence that they had been attacked by the
  Sudanese government and allies.
• Troll
  Patrol:
  Quantitative analysis of online abuse and violence against UK and US
  women on Twitter.

This is definitely a talk I would recommend everybody to watch and you
can do so from here!

Tour Toulouse!

As I was only heading home on the following Monday, I had the entire
weekend to explore Toulouse! I was staying near the Capitole and as
Toulouse is pretty walkable I didn’t have to use public transportation
at all during my stay. I think I just about walked every street in the
city center! Unfortunately, the Musee de Augustins was closed but I was
able to visit most of the other sites! Below are some pictures:

Sunday was also Bastille Day so there were some fireworks on display as
well. All in all I had a great time in Toulouse!

Conclusion

This was my second useR! Conference and I enjoyed it quite a lot, not to
mention I got to do some sightseeing which I wasn’t able to do much of
in Brisbane last year. I met a lot of people that I follow on Twitter
and I’ve had people come up to me who recognized me from all the data
viz/blog posts I do (a first for me) which was really cool (and it helps
as I’m very nervous about approaching people especially since they are
usually surrounded by other people and I don’t want to interrupt their
conversation and… “Oh no it’s time for the next session!”, etc.)!

During a post-conference dinner I had with a dozen or so random
R users that were still in Toulouse (including Diane Cook, Will Chase, Saras Windecker, Irene Steves, Alimi Eyitayo among others – and some that I didn’t even get to talk to because our group was so big) we all talked about how important the
community is. With how open everything is in regards to the talks
being recorded and the materials being put online you don’t necessarily
have to come all the way to the conference to be able to learn the material.
However, the important component of these conferences is being able to talk to the people and engaging with the community which
is something I’ve really felt to be a part of since I started R and
going to conferences in the past 2 years or so. I think nearly each one
of the people I sat with on the table at dinner that night came from a
different country and worked in completely different areas which made
for some real eye-opening discussion about how R is used worldwide and across industries. I
also learned about cultural differences in tech, especially women in tech
in Nigeria from Alimi Eyitayo (who
also gave a talk on Scaling useR Communities with Engagement and
Retention
Models at the
conference).

There were still a ton of great talks I missed so I’m excited to watch the rest on Youtube. I think I will be at RStudio::Conference next year in San Francisco so hopefully I’ll see some of you there!

Continue Reading…

Collapse

Read More

Forecast KPI: RMSE, MAE, MAPE and Bias

Inferential Statistics: Understanding Hypothesis Testing Using Chi-Square Test

Harvard Data Science Review – A Microscopic, Telescopic, and Kaleidoscopic View of Data Science

A Turbulent Year: The 2019 Data & AI Landscape

Part II: Major Trends in the 2019 Data & AI Landscape

The data that trains AI increasingly calls into question AI

AI analyzed 3.3 million scientific abstracts and discovered possible new materials

Object-oriented programming for data scientists: Build your ML estimator

A Basic NLP Tutorial for News Multiclass Categorization

A Simple Framework for Designing Choices

Statistics for people in a hurry

ONNX

BERT Explained: State of the art language model for NLP

The Scariest Thing About DeepNude Wasn’t the Software

Continue Reading…

Collapse

Read More

Article: Conversational AI ? but where is the I?

Paper: Task Selection Policies for Multitask Learning

Paper: Counterfactual Reasoning for Fair Clinical Risk Prediction

Paper: To Model or to Intervene: A Comparison of Counterfactual and Online Learning to Rank from User Interactions

Paper: A Causal Bayesian Networks Viewpoint on Fairness

Paper: Posterior Predictive Treatment Assignment Methods for Causal Inference in the Context of Time-Varying Treatments

Continue Reading…

Collapse

Read More

“There will be no AI worthy of the name without causal inference.” Miguel A. Hernán, John Hsu, Brian Healy ( July 12, 2018 )

Continue Reading…

Collapse

Read More

Bonsai
Extreme multi-label classification refers to supervised multi-label learning involving hundreds of thousand or even millions of labels. In this paper, we develop a shallow tree-based algorithm, called Bonsai, which promotes diversity of the label space and easily scales to millions of labels. Bonsai relaxes the two main constraints of the recently proposed tree-based algorithm, Parabel, which partitions labels at each tree node into exactly two child nodes, and imposes label balanced-ness between these nodes. Instead, Bonsai encourages diversity in the partitioning process by (i) allowing a much larger fan-out at each node, and (ii) maintaining the diversity of the label set further by enabling potentially imbalanced partitioning. By allowing such flexibility, it achieves the best of both worlds – fast training of tree-based methods, and prediction accuracy better than Parabel, and at par with one-vs-rest methods. As a result, Bonsai outperforms state-of-the-art one-vs-rest methods such as DiSMEC in terms of prediction accuracy, while being orders of magnitude faster to train. The code for \bonsai is available at https://…/bonsai. …

HybridNet
In this paper, we introduce a new model for leveraging unlabeled data to improve generalization performances of image classifiers: a two-branch encoder-decoder architecture called HybridNet. The first branch receives supervision signal and is dedicated to the extraction of invariant class-related representations. The second branch is fully unsupervised and dedicated to model information discarded by the first branch to reconstruct input data. To further support the expected behavior of our model, we propose an original training objective. It favors stability in the discriminative branch and complementarity between the learned representations in the two branches. HybridNet is able to outperform state-of-the-art results on CIFAR-10, SVHN and STL-10 in various semi-supervised settings. In addition, visualizations and ablation studies validate our contributions and the behavior of the model on both CIFAR-10 and STL-10 datasets. …

Sure Thing Principle (STP)
In 1954, Jim Savage introduced the Sure Thing Principle to demonstrate that preferences among actions could constitute an axiomatic basis for a Bayesian foundation of statistical inference. Here, we trace the history of the principle, discuss some of its nuances, and evaluate its significance in the light of modern understanding of causal reasoning. The sure-thing principle (STP) was introduced by L.T. Savage using the following story: ‘A businessman contemplates buying a certain piece of property. He considers the outcome of the next presidential election relevant. So, to clarify the matter to himself, he asks whether he would buy if he knew that the Democratic candidate were going to win, and decides that he would. Similarly, he considers whether he would buy if he knew that the Republican candidate were going to win, and again finds that he would. Seeing that he would buy in either event, he decides that he should buy, even though he does not know which event obtains, or will obtain, as we would ordinarily say.’ …

BayesNAS
One-Shot Neural Architecture Search (NAS) is a promising method to significantly reduce search time without any separate training. It can be treated as a Network Compression problem on the architecture parameters from an over-parameterized network. However, there are two issues associated with most one-shot NAS methods. First, dependencies between a node and its predecessors and successors are often disregarded which result in improper treatment over zero operations. Second, architecture parameters pruning based on their magnitude is questionable. In this paper, we employ the classic Bayesian learning approach to alleviate these two issues by modeling architecture parameters using hierarchical automatic relevance determination (HARD) priors. Unlike other NAS methods, we train the over-parameterized network for only one epoch then update the architecture. Impressively, this enabled us to find the architecture in both proxy and proxyless tasks on CIFAR-10 within only 0.2 GPU days using a single GPU. As a byproduct, our approach can be transferred directly to compress convolutional neural networks by enforcing structural sparsity which achieves extremely sparse networks without accuracy deterioration. …

Continue Reading…

Collapse

Read More

Rajesh Venkatachalapathy writes:

I recently came across a world bank document claiming that gendered languages reduce women’s labor force participation rates. It is summarized in the following press release: Gendered Languages May Play a Role in Limiting Women’s Opportunities, New Research Finds.

This sounds a lot like the piranha problem, if there is any effect at all.

I [Venkatachalapathy] am disturbed by claims of large effects in their study. Their work seems to rely conceptually on the Sapir-Whorf hypothesis in linguistics, which is also quiet controversial on its own. I am curious to know what your take is on this report.

He continues:

The cognitive science behind Sapir-Whorf, and the related field of embodied cognition in general is quiet controversial; it appeals to so many people, yet has very weak evidence (see for example, the recent book by McWhorter). This paper seems to magnify this to say something so strong about macroeconomic labor market demographic indicators. I cannot avoid comparisons with Pinker’s hypothesis in his most recent book that enlightenment thought and secular humanistic principles derived from it has been one of the primary drivers of the civilizing process of the Norbert Elias kind or the Pinker kind.

I am not claiming that such macro-level claims can never be justified. For example, I just began reading your academic colleague, economist Suresh Naidu’s recent paper on how democratization in countries causes economic growth. From the looks of it, they seem to have worked hard at establishing their main hypothesis. Maybe, their [Naidu or his collaborators] approach might provide us with additional insight on whether the causal claims of the paper on gendered language and workforce participation is reasonable and defensible with existing data, and with their [the paper’s] data analysis approach. I just find it difficult to imagine how a psychologically weak effect can suddenly become magnified when scaled to level of large scale societies.

After having trained hard to be skeptical of all causal claims over the years, I see what I feel is an epidemic of causal claims popping up in the literature and I find it hard to believe them all, especially given the fact that progress in philosophical causality and causal inference has been only incremental.

My response: I agree that such claims from observational data in cross-country and cross-cultural comparisons can be artifactual, and languages are correlated with all sorts of things. I don’t know enough about the topic to say more.

Continue Reading…

Collapse

Read More

1. Researchers solve the Rubik’s cube puzzle using machine learning (deep learning).
2. There has been a rise in the popularity of “deep learning” following some major breakthroughs in tasks like image recognition. Yet, at least as far as recommender systems are concerned, there are reasons to be skeptical of the good results being reported:
   

   In this work, we report the results of a systematic analysis of algorithmic proposals for top-n recommendation tasks. Specifically, we considered 18 algorithms that were presented at top-level research conferences in the last years. Only 7 of them could be reproduced with reasonable effort. For these methods, it however turned out that 6 of them can often be outperformed with comparably simple heuristic methods, e.g., based on nearest-neighbor or graph-based techniques. The remaining one clearly outperformed the baselines but did not consistently outperform a well-tuned non-neural linear ranking method. Overall, our work sheds light on a number of potential problems in today’s machine learning scholarship and calls for improved scientific practices in this area.

3. Your blood contains about four grams of glucose/sugar (it is a tiny amount).
4. Our brain oscillates at a rate of about 40 Hz (40 times per second). Some researchers may have found the cells responsible for coordinating these waves.
5. Are we suffering from record-setting heat waves? A recent American government report concludes that we are not:
   

   (…) the warmest daily temperature of the year increased in some parts of the West over the past century, but there were decreases in almost all locations east of the Rocky Mountains. In fact, all eastern regions experienced a net decrease (…), most notably the Midwest (about 2.2°F) and the Southeast (roughly 1.5°F). (…) As with warm daily temperatures, heat wave magnitude reached a maximum in the 1930s.

   The same report observes that cold extremes have become less common, however.

Continue Reading…

Collapse

Read More

Understanding who you are is strongly related to understanding your family history. Discovering ancestors is now a popular hobby, as many archives are available on the internet. The GEDCOM format provides a standardised way to store information about ancestors. This article shows how to develop a GEDCOM reader using the R language.

The GEDCOM format

The GEDCOM format is not an ideal way to store information, but it has become the de-facto standard for family history. This format includes metadata and two sets of data. The file contains a list of the individuals, and it lists the families to which they belong.

The basic principle is that each line has a level, indicated by the first digit. At level zero, we find metadata and the individuals and their family. At level one, we see the various types of data, such as births, deaths and marriages. The deeper levels provide the data for these events.

Heiner Eichmann maintains a website that explains the format and provides some examples of files to help you understand the syntax.

The GEDCOM format is not only old in the way it stores data, but it is also limited in the types of human relationships. These files only store genetic relationships between people and assume that these relationships are marriages between a wife and a husband. Human relationships are, however, a lot more complicated than the genetic relationships between children and their parents, grandparents and ancestors.

These issues aside, all genealogical software can export a file to GEDCOM. The next section shows how to create a basic GEDCOM reader using the stringr, tibble and dplyr packages from the Tidyverse.

GEDCOM Reader

The read.gedcom() function takes a GEDCOM file as input and delivers a data frame (tibble) with basic information:

• ID
• Full name
• Gender
• Birthdate and place
• Father
• Mother
• Death date and place

This code only can be easily expanded to include further fields by adding lines in the while-loops and including the fields in the data frame.

The first lines read the file and setup the data frame. The extract() function extracts an individual’s ID from a line in the file. The for loop runs through each line of the GEDCOM file. When the start of a new individual is detected, the GEDCOM reader collects the relevant information.

Births and christenings are considered equal to simplify the data. In older data, we often only know one or the other. The function looks for the start of a family. It extracts the husband and wife and assigns these as parents to each of the children. The last section cleans the data and returns the result.

## Read GEDCOM file

## The Devil is in the Data
## lucidmanager.org/data-science
## Dr Peter Prevos

read.gedcom <- function(gedcom.loc) {
    require(stringr)
    require(tibble)
    require(dplyr)

    gedcom <- str_squish(readLines(gedcom.loc))
    idv <- sum(grepl("^0.*INDI$", gedcom))
    fam <- sum(grepl("^0.*FAM$", gedcom))
    cat(paste("Individuals: ", idv, "\n"))
    cat(paste("Families: ", fam, "\n"))
    family <- tibble(id = NA,
                     Full_Name = NA,
                     Gender = NA,
                     Birth_Date = NA,
                     Birth_Place = NA,
                     Father_id = NA,
                     Mother_id = NA,
                     Death_Date = NA,
                     Death_Place = NA)
    ## Extract data
    extract <- function(line, type) {
        str_trim(str_sub(line, str_locate(line, type)[2] + 1))
    }
    id <- 0
    for (l in 1:length(gedcom)) {
        if (str_detect(gedcom[l], "^0") & str_detect(gedcom[l], "INDI$")) {
            id <- id + 1
            family[id, "id"] <- unlist(str_split(gedcom[l], "@"))[2]
            l <- l + 1
            while(!str_detect(gedcom[l], "^0")) {
                if (grepl("NAME", gedcom[l]))
                    family[id, "Full_Name"] <- extract(gedcom[l], "NAME")
                if (grepl("SEX", gedcom[l]))
                    family[id, "Gender"] <- extract(gedcom[l], "SEX")
                l <- l + 1
                if (grepl("BIRT|CHR", gedcom[l])) {
                    l <- l + 1
                    while (!str_detect(gedcom[l], "^1")) {
                        if (grepl("DATE", gedcom[l]))
                            family[id, "Birth_Date"] <- extract(gedcom[l], "DATE")
                        if (grepl("PLAC", gedcom[l]))
                            family[id, "Birth_Place"] <- extract(gedcom[l], "PLAC")
                        l <- l + 1
                    }
                }
                if (grepl("DEAT|BURI", gedcom[l])) {
                    l <- l + 1
                    while (!str_detect(gedcom[l], "^1")) {
                        if (grepl("DATE", gedcom[l]))
                            family[id, "Death_Date"] <- extract(gedcom[l], "DATE")
                        if (grepl("PLAC", gedcom[l]))
                            family[id, "Death_Place"] <- extract(gedcom[l], "PLAC")
                        l <- l + 1
                    }
                }
            }
        }
        if (str_detect(gedcom[l], "^0") & str_detect(gedcom[l], "FAM")) {
            l <- l + 1
            while(!str_detect(gedcom[l], "^0")) {
                if (grepl("HUSB", gedcom[l]))
                    husband <- unlist(str_split(gedcom[l], "@"))[2]
                if (grepl("WIFE", gedcom[l]))
                    wife <- unlist(str_split(gedcom[l], "@"))[2]
                if (grepl("CHIL", gedcom[l])) {
                    child <- which(family$id == unlist(str_split(gedcom[l], "@"))[2])
                    family[child, "Father_id"] <- husband
                    family[child, "Mother_id"] <- wife
                }
                l <- l + 1
            }
        }
    }
    family %>%
        mutate(Full_Name = gsub("/", "", str_trim(Full_Name)),
               Birth_Date = as.Date(family$Birth_Date, format = "%d %b %Y"),
               Death_Date = as.Date(family$Death_Date, format = "%d %b %Y")) %>%
        return()
}

Analysing the data

There are many websites with GEDCOM files of family histories of famous and not so famous people. The Famous GEDCOMs website has a few useful examples to test the GEDCOM reader.

Once the data is in a data frame, you can analyse it any way you please. The code below downloads a file with the presidents of the US, with their ancestors and descendants. The alive() function filters people who are alive at a certain date. For people without birth date, it sets a maximum age of 100 years.

The histogram shows the distribution of ages at time of death of all the people in the presidents file.

These are just some random examples of how to analyse family history data with this GEDCOM reader. The next article will explain how to plot a population pyramid using this data. A future article will discuss how to visualise the structure of family history.

Birth years of people of in the Presidents file

## Basic family history statistics
library(tidyverse)
library(lubridate)
source("read.gedcom.R")
presidents <- read.gedcom("https://webtreeprint.com/tp_downloader.php?path=famous_gedcoms/pres.ged")

filter(presidents, grepl("Jefferson", Full_Name))

mutate(presidents, Year = year(Birth_Date)) %>%
    ggplot(aes(Year)) +
    geom_histogram(binwidth = 10, fill = "#6A6A9D", col = "white") +
    labs(title = "Birth years in the presidents file")
ggsave("../../Genealogy/years.png")

alive <- function(population, census_date){
    max_date <- census_date + 100 * 365.25
    filter(people, (is.na(Birth_Date) & (Death_Date <= max_date &
                                         Death_Date >= census_date)) |
                   (Birth_Date <= census_date & Death_Date >= census_date)) %>%
        arrange(Birth_Date) %>%
        mutate(Age = as.numeric(census_date - Birth_Date) / 365.25) %>%
        return()
}

alive(presidents, as.Date("1840-03-07"))

The post GEDCOM Reader for the R Language: Analysing Family History appeared first on The Lucid Manager.

Continue Reading…

Collapse

Read More

This year, the greater R community gathering useR! took place in sunny Toulouse in July, bringing together over 1000 practitioners from both academia and industry.

The event spanned over five days, including:

• a tidyverse day
• one full day of workshops
• 6 keynotes and a few sponsor talks
• contributed talks and lightning talks over 6 parallel sessions
• a poster session
• a rich social program.

Here are a few impressions from Francesca, Nikki and Riccardo who attended the conference on behalf of Mirai Solutions.

Satellite Events

On Monday, ahead of the main conference, multiple satellite activities were available for the R aficionados already in town.

Forwards organized a series of meetings between forwards friends, diversity scholarship awardees and the “newbies”.

A tidyverse developer day also took place at the conference venue, where attendees could work on tidyverse-related issues, learn about coding best practices and documentation, as well as being exposed to git and the development workflow.

Tutorial Day

An extremely rich offer of tutorials, with two blocks of nine parallel sessions, made it very difficult to make up our mind and choose where to go. Topics ranged from development workflows to visualization, Shiny, machine learning, data analysis and modeling. This showed once again how diverse the R language is!

The day closed with a welcome apero hosted by the Mayor of Toulouse in the City Hall and a hilarious translation from French to nerd-English by Romain François.

First Conference Day

What better start than with Music? The useR! 2019 organizers set the mood right by hosting the performance of Moultaqa Salam (“Meet and peace”), a group melting perfectly the different souls and sounds of South-West Europe and North Africa.

Julia Stewart Lowndes, aka @juliesquid opened with an inspiring talk about open source and collaboration. R can be The Force to do better science in less time and Twitter can help building the community where sharing and communication can take place.

The famous wizard Hadley Wickham gave a talk about the latest enhancements of data tidying, proving that he does not just live in the internet, but also attends conferences!

Next a unicorn made it to the stage to show us the many little tricks hidden in dplyr. He had a very nice little helper, which reminds us all of the importance of supporting working parents with the potential difficulties they may face.

Miraier Riccardo gave a talk on Shiny app deployment and integration into a custom website gallery, where he presented the challenges and steps for embedding and integrating a Shiny app into an existing website. As an example he used SmaRP, a Shiny app designed to guide people working in Switzerland towards a strategic retirement plan, previously presented at eRum in 2018.

Shiny has been a prominent topic with two sessions covering aspects ranging from Shiny development workflows with package golem to scalable, enterprise-level applications.

Julie Josse closed the day’s work with an academic-flavoured presentation about her research on dealing with missing numbers. However the fun continued with the conference dinner, taking place at the amazing Cité de l’Espace, where we had a lovely time gazing at the interesting exhibitions and tasting nice French specialties!

Second Conference Day

Joe Cheng’s opening keynote premiered the brand new shinymeta, a R package tackling the issue of reproducibility with Shiny by providing tools for capturing logic in a Shiny app and exposing it as code that can be run outside of Shiny.

In the lightning talks session on “Workflow and Development”, Miraier Nikki presented CompareWith, a Meld-based R package, which provides user-friendly RStudio addins to perform diff and merge tasks. The slides from her talk can be found here.

The two “Programming” sessions were a great source of updates and novelty. Of particular interest were the introduction of the new embracing operator to simplify creating tidy eval functions, and Davis Vaughan’s talk about the new package rray that makes array calculations much easier. Colin Gillespie’s talk about Security and R was also one of the highlights.

The very interesting session on “Community and Conferences” work gave a nice overview of how R is spreading in Africa and Latin America. It also featured a panel on the truth about satRdays, which offered a set of useful tips for organizing a (successful) event.

Closing the day was a poster session with a nice apero. It was extremely fun to first see the presenters advertising their posters in a 30 seconds time-slot, some giving directions to the location of their poster, some having to step off stage while still talking. Overall a great occasion to network and get a glimpse of various interesting projects.

Third Conference Day

The last day of useR! 2019 featured a last round of lightning and regular talks. Many notable presentations were in the “Performance” session, which included interesting updates about packages data.table
and future. A new promising package pak for managing package installation and dependencies was introduced by Gábor Csárdi, whereas Jim Hester showed how package vroom can boost importing large data-sets.

The closing keynote by Julien Cornebise about ‘AI for Good’ in the R and Python ecosystems conveyed an extremely powerful message on how to use data science for good, e.g. to measure violence and abuse against women on Twitter or find burned villages in Darfur via satellite images. Checking out his talk is highly recommended.

Finally Heather Turner got awarded for her central role in the R community and we would like to add our voice in praising her commitment.

Closing Remarks

If you could not attend a talk or would like to listen to one again, video recordings of the presentations are (will be) available online on the R Consortium Youtube channel, while slides of the talks are downloadable from the useR! 2019 website.

Overall it was a very enjoyable and insightful event and a great venue in Toulouse. We look forward to useR! 2020 in St. Louis! Before that, and closer for Europeans, we’ll be sure to attend eRum2020, which Mirai Solutions is also supporting actively and takes place in Milan May 27th-30th.

Continue Reading…

Collapse

Read More

AI sees itself via Humans

Opening Black Boxes: How to leverage Explainable Machine Learning

A Simple Neural Attentive Meta-Learner – SNAIL

Anomaly Detection in Images

Building a Neural Network in Python – Language Modeling Task

Reinforcement Learning – Model Based Planning Methods Extension

Multilingual Universal Sentence Encoder for Semantic Retrieval

PCA: Eigenvectors and Eigenvalues

Deep Learning for Survival Analysis

Log Book – Guide to Distance Measuring Approaches for K- Means Clustering

A Comparison Study on Similarity and Dissimilarity Measures in Clustering Continuous Data

Tracking ML Experiments using MLflow

Simulating Data in R: Examples in Writing Modular Code

Maximizing group happiness in White Elephants using the Hungarian optimal assignment algorithm

Recent Advances in Modern Computer Vision

Continue Reading…

Collapse

Read More

• An Exploratory Study of Live-Streamed Programming
• Feature Toggle Driven Development: Practices usedby Practitioners
• Improving the Projection of Global Structures in Data through Spanning Trees
• Towards Optimal Moment Estimation in Streaming and Distributed Models
• Augmenting Neural Nets with Symbolic Synthesis: Applications to Few-Shot Learning
• Differentiable Bayesian Neural Network Inference for Data Streams
• MIPaaL: Mixed Integer Program as a Layer
• An Evolutionary Algorithm of Linear complexity: Application to Training of Deep Neural Networks
• BeSense: Leveraging WiFi Channel Data and Computational Intelligence for Behavior
• A Study and Analysis of a Feature Subset Selection Technique using Penguin Search Optimization Algorithm (FS-PeSOA)
• Distributed Kalman Filtering under Model Uncertainty
• k-hop Graph Neural Networks
• Using dynamic routing to extract intermediate features for developing scalable capsule networks
• Bringing Giant Neural Networks Down to Earth with Unlabeled Data
• Relational Memory-based Knowledge Graph Embedding
• Understanding Deep Learning Techniques for Image Segmentation
• Continuous-Time Markov Decision Processes with Controlled Observations
• Tackling Graphical NLP problems with Graph Recurrent Networks
• Satellite System Graph: Towards the Efficiency Up-Boundary of Graph-Based Approximate Nearest Neighbor Search
• On substitutions closed under derivation: examples
• Secrecy Analysis and Learning-based Optimization of Cooperative NOMA SWIPT Systems
• Proceedings of FACTS-IR 2019
• Automated Word Stress Detection in Russian
• Curvature-dimension conditions for diffusions under time change
• Learning to Handle Parameter Perturbations in Combinatorial Optimization: an Application to Facility Location
• Equal bi-Vectorized (EBV) method to high performance on GPU
• Deep Learning-powered Iterative Combinatorial Auctions
• The Importance of Telescope Training in Data Interpretation
• Exploration by Optimisation in Partial Monitoring
• Structured inversion of the Bernstein mass matrix
• Hello, It’s GPT-2 — How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems
• Path Weights in Concentration Graphs
• Unsupervised Adversarial Attacks on Deep Feature-based Retrieval with GAN
• ACTNET: end-to-end learning of feature activations and aggregation for effective instance image retrieval
• Convergent discretisation schemes for transition path theory for diffusion processes
• Asymptotics for Spherical Functional Autoregressions
• Coulomb gases under constraint: some theoretical and numerical results
• Relevance of topological disorder on the directed percolation phase transition
• On the Correlation of Critical Points and Angular Trispectrum for Random Spherical Harmonics
• Girls Create Games: Lessons Learned
• Characterization of $k$-spectrally monomorphic Hermitian $l2$-structures
• Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Camera
• Asynchronous Majority Dynamics in Preferential Attachment Trees
• The Algebra of Schur Operators
• Patterns in sets of positive density in trees and affine buildings
• Signal Conditioning for Learning in the Wild
• Dual Extrapolation for Sparse Generalized Linear Models
• Concentration inequalities for random matrix products
• Equiprobable mappings in weighted constraint grammars
• Exponential convergence to equilibrium in supercritical kinetically constrained models at high temperature
• Kingman’s coalescent with erosion
• The University of Edinburgh’s Submissions to the WMT19 News Translation Task
• On a Generalization of the Marriage Problem
• Kinetics of thermal Mott transitions in the Hubbard model
• Voice Pathology Detection Using Deep Learning: a Preliminary Study
• Fast, higher-order direct/iterative hybrid solver for scattering by Inhomogeneous media — with application to high-frequency and discontinuous refractivity problems
• Functional treatment of asymmetric copulas
• Gesture-to-Gesture Translation in the Wild via Category-Independent Conditional Maps
• Posit NPB: Assessing the Precision Improvement in HPC Scientific Applications
• Estimating densities with nonlinear support using Fisher-Gaussian kernels
• Dynamically orthogonal tensor methods for high-dimensional nonlinear PDEs
• On the Price of Anarchy of Cost-Sharing in Real-Time Scheduling Systems
• Predicting phenotypes from microarrays using amplified, initially marginal, eigenvector regression
• A machine learning framework for computationally expensive transient models
• Properties for the Frechet Mean in Billera-Holmes-Vogtmann Treespace
• Planar Disjoint Paths in Linear Time
• Multilevel models for continuous outcomes
• The spectral matrices associated with the stochastic Darboux transformations of random walks on the integers
• Online-Learning for min-max discrete problems
• NH-TTC: A gradient-based framework for generalized anticipatory collision avoidance
• On the residues vectors of a rational class of complex functions. Application to autoregressive processes
• Pykaldi2: Yet another speech toolkit based on Kaldi and Pytorch
• Characterising random partitions by random colouring
• Efficient average-case population recovery in the presence of insertions and deletions
• Random Khatri-Rao-Product Codes for Numerically-Stable Distributed Matrix Multiplication
• Upper bounds for inverse domination in graphs
• Multi-Hop Wireless Optical Backhauling for LiFi Attocell Networks: Bandwidth Scheduling and Power Control
• Linear programming bounds for cliques in Paley graphs
• The integration of variable generation and storage into electricity capacity markets
• Resolvability of Hamming Graphs
• Entropy-stable discontinuous Galerkin approximation with summation-by-parts for the incompressible Navier-Stokes equations with variable density and artificial compressibility
• Convergence Analysis of Machine Learning Algorithms for the Numerical Solution of Mean Field Control and Games: I — The Ergodic Case
• Coloring invariants of knots and links are often intractable
• Learning Complex Basis Functions for Invariant Representations of Audio
• Distributed Black-Box Optimization via Error Correcting Codes
• On the Differential Linear Connectivity Table of Vectorial Boolean Functions
• ND-Wavelets Derived from Anti-symmetric Systems of Isolated Particles using the Determinant of Slater
• Local Distribution Obfuscation via Probability Coupling
• The Two-Sided Game of Googol and Sample-Based Prophet Inequalities
• Intelligent Reflecting Surface: Practical Phase Shift Model and Beamforming Optimization
• Fisher-Rao Geometry and Jeffreys Prior for Pareto Distribution
• SynthText3D: Synthesizing Scene Text Images from 3D Virtual Worlds
• The automorphism groups of some token graphs
• The Futility of Bias-Free Learning and Search
• Extracting Interpretable Physical Parameters from Spatiotemporal Systems using Unsupervised Learning
• Efficient methods to determine the reversibility of general 1D linear cellular automata in polynomial complexity
• Motion Planning Networks: Bridging the Gap Between Learning-based and Classical Motion Planners
• A Conditional Wasserstein Generative Adversarial Network for Pixel-level Crack Detection using Video Extracted Images
• Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition
• Combinatorics in the exterior algebra and the Bollobás Two Families Theorem
• Shape optimization for composite materials and scaffolds
• Multiscale Principle of Relevant Information for Hyperspectral Image Classification
• Structure-Aware Residual Pyramid Network for Monocular Depth Estimation
• Gait Generation using Intrinsically Stable MPC in the Presence of Persistent Disturbances
• On the convergence rate of some nonlocal energies
• Minimal Sample Subspace Learning: Theory and Algorithms
• Perfect sampling from spatial mixing
• Towards Characterizing and Limiting Information Exposure in DNN Layers
• M3D-RPN: Monocular 3D Region Proposal Network for Object Detection
• Energy-Efficient Radio Resource Allocation for Federated Edge Learning
• Cross-Lingual Transfer Learning for Question Answering
• Multi-Element Long Distance Dependencies: Using SPk Languages to Explore the Characteristics of Long-Distance Dependencies
• Learning better generative models for dexterous, single-view grasping of novel objects
• Lower Bound for RIP Constants and Concentration of Sum of Top Order Statistics
• On the two-dimensional hyperbolic stochastic sine-Gordon equation
• Aggregate-Eliminate-Predict: Detecting Adverse Drug Events from Heterogeneous Electronic Health Records
• One-dimensional number-conserving cellular automata
• Image Evolution Trajectory Prediction and Classification from Baseline using Learning-based Patch Atlas Selection for Early Diagnosis
• The Use of Gaussian Processes in System Identification
• ALFA: Agglomerative Late Fusion Algorithm for Object Detection
• Efficient self-stabilizing leader election in population protocols
• S&CNet: A Enhanced Coarse-to-fine Framework For Monocular Depth Completion
• A new approach to Poissonian two-armed bandit problem
• Changing Views: Persuasion Modeling and Argument Extraction from Online Discussions
• Evolvability ES: Scalable and Direct Optimization of Evolvability
• Adaptive Context Encoding Module for Semantic Segmentation
• On a degeneracy ratio for bounded convex polytopes
• Skew Generalized Cyclic Code over $ R[x_1;σ_1,δ_1][x_2;σ_2,δ_2]$
• On Discrete-Time/Frequency-Periodic End-to-End Fiber-Optical Channel Models
• Parameterized Exploration
• Motion Segmentation Using Locally Affine Atom Voting
• Dogfooding: use IBM Cloud services to monitor IBM Cloud infrastructure
• Automated Gaming Pommerman: FFA
• Seeker based Adaptive Guidance via Reinforcement Meta-Learning Applied to Asteroid Close Proximity Operations
• Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysis
• Supporting Security Sensitive Tenants in a Bare-Metal Cloud
• Speaker Recognition with Random Digit Strings Using Uncertainty Normalized HMM-based i-vectors
• BUT VOiCES 2019 System Description
• Highly neighborly centrally symmetric spheres
• Inference for high-dimensional linear mixed-effects models: A quasi-likelihood approach
• A semi-Lagrangian discontinuous Galerkin (DG) — local DG method for solving convection-diffusion-reaction equations
• Online Rental Housing Market Representation and the Digital Reproduction of Urban Inequality
• Preselection Bandits under the Plackett-Luce Model
• Towards Robust Voice Pathology Detection
• Information Pollution by Social Bots
• An Assumption-Free Exact Test For Fixed-Design Linear Models With Exchangeable Errors
• FMRI data augmentation via synthesis
• Stochastic Convergence Results for Regularized Actor-Critic Methods
• Bilevel Optimization and Variational Analysis
• A Real-Time mmWave Communication Testbed with Phase Noise Cancellation
• Neural Embedding for Physical Manipulations
• Leveraging Auxiliary Information on Marginal Distributions in Nonignorable Models for Item and Unit Nonresponse
• ThirdEye: Triplet Based Iris Recognition without Normalization
• Langevin method for a continuous stochastic car-following model and its stability conditions
• From quadratic Hawkes processes to super-Heston rough volatility models with Zumbach effect
• A Versatile Software Systolic Execution Model for GPU Memory-Bound Kernels
• Zeros of ferromagnetic 2-spin systems

Continue Reading…

Collapse

Read More

Improving the Projection of Global Structures in Data through Spanning Trees

Towards Optimal Moment Estimation in Streaming and Distributed Models

One of the oldest problems in the data stream model is to approximate the -th moment of an underlying vector , which is presented as a sequence of poly updates to its coordinates. Of particular interest is when . Although a tight space bound of bits is known for this problem when both positive and negative updates are allowed, surprisingly there is still a gap in the space complexity when all updates are positive. Specifically, the upper bound is bits, while the lower bound is only bits. Recently, an upper bound of bits was obtained assuming that the updates arrive in a random order. We show that for , the random order assumption is not needed. Namely, we give an upper bound for worst-case streams of bits for estimating . Our techniques also give new upper bounds for estimating the empirical entropy in a stream. On the other hand, we show that for , in the natural coordinator and blackboard communication topologies, there is an bit max-communication upper bound based on a randomized rounding scheme. Our protocols also give rise to protocols for heavy hitters and approximate matrix product. We generalize our results to arbitrary communication topologies , obtaining an max-communication upper bound, where is the diameter of . Interestingly, our upper bound rules out natural communication complexity-based approaches for proving an bit lower bound for for streaming algorithms. In particular, any such lower bound must come from a topology with large diameter.

Augmenting Neural Nets with Symbolic Synthesis: Applications to Few-Shot Learning

Differentiable Bayesian Neural Network Inference for Data Streams

MIPaaL: Mixed Integer Program as a Layer

An Evolutionary Algorithm of Linear complexity: Application to Training of Deep Neural Networks

The performance of deep neural networks, such as Deep Belief Networks formed by Restricted Boltzmann Machines (RBMs), strongly depends on their training, which is the process of adjusting their parameters. This process can be posed as an optimization problem over n dimensions. However, typical networks contain tens of thousands of parameters, making this a High-Dimensional Problem (HDP). Although different optimization methods have been employed for this goal, the use of most of the Evolutionary Algorithms (EAs) becomes prohibitive due to their inability to deal with HDPs. For instance, the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) which is regarded as one of the most effective EAs, exhibits the enormous disadvantage of requiring memory and operations, making it unpractical for problems with more than a few hundred variables. In this paper, we introduce a novel EA that requires operations and memory, but delivers competitive solutions for the training stage of RBMs with over one million variables, when compared against CMA-ES and the Contrastive Divergence algorithm, which is the standard method for training RBMs.

BeSense: Leveraging WiFi Channel Data and Computational Intelligence for Behavior

A Study and Analysis of a Feature Subset Selection Technique using Penguin Search Optimization Algorithm (FS-PeSOA)

Distributed Kalman Filtering under Model Uncertainty

k-hop Graph Neural Networks

Using dynamic routing to extract intermediate features for developing scalable capsule networks

Bringing Giant Neural Networks Down to Earth with Unlabeled Data

Relational Memory-based Knowledge Graph Embedding

Understanding Deep Learning Techniques for Image Segmentation

Continuous-Time Markov Decision Processes with Controlled Observations

Tackling Graphical NLP problems with Graph Recurrent Networks

Satellite System Graph: Towards the Efficiency Up-Boundary of Graph-Based Approximate Nearest Neighbor Search

Continue Reading…

Collapse

Read More

While I was browsing the website of fellow R blogger Ryo Nakagawara¹, I
was intrigued by his “Visualizations” page.
The concept of creating an online “portfolio” is not novel ², but
I hadn’t thought to make one as a compilation of my own work (from blog posts)…
until now .

The code that follows shows how I generated the
body of my visualization portfolio page.
The task is achieved in a couple of steps.

1. Identify the file path of each blog post in my local blog folder

2. For each post, extract the date and title of the blog post from the front matter, as well as the name and links to the image files.

3. Combine the extracted information into a character vector that can be copy-pasted to a gallery page.

I should state a couple of caveats/notes for anyone looking to emulate my approach.

• I take advantage of the fact that I use the same prefix for (almost) all
  visualization that I generate with R—viz_—³, as well as the same file
  format—.png.

• At the time of writing, my website—a {blogdown}-based
  website—uses Hugo’s page bundles
  content organization system,
  as well as the with the popular Academic theme for Hugo.
  Thus, there’s no guarantee that the following code will work for you “as is”. ⁴
  (If it doesn’t work as is, I think modifying the code should be fairly straightforward.)

• I create headers and links from the titles of the blog posts
  (via something like sprintf('## %s, [%s](%s)', ...) and I order everything according
  to descending date and ascending line in the post.
  This may not be what you would like for your gallery format.

library(tidyverse)

paths_post_raw <-
  fs::dir_ls(
    'content/post/',
    regexp = 'index[.]md$',
    recurse = TRUE
  ) %>% 
  # Ignore the "_index.md" at the base of the content/post directory.
  # Would need to also ignore draft posts if there are draft posts.
  str_subset('_index', negate = TRUE)
paths_post_raw[1:10]

##  [1] "content/post/analysis-texas-high-school-academics-1-intro/index.md"        
##  [2] "content/post/analysis-texas-high-school-academics-2-competitions/index.md" 
##  [3] "content/post/analysis-texas-high-school-academics-3-individuals/index.md"  
##  [4] "content/post/analysis-texas-high-school-academics-4-schools/index.md"      
##  [5] "content/post/analysis-texas-high-school-academics-5-miscellaneous/index.md"
##  [6] "content/post/cheat-sheet-rmarkdown/index.md"                               
##  [7] "content/post/data-science-podcasts/index.md"                               
##  [8] "content/post/dry-principle-make-a-package/index.md"                        
##  [9] "content/post/gallery-visualizations/old/index.md"                          
## [10] "content/post/interval-data-nycflights13/index.md"

# Define some important regular expressions (or "regex"es).
# These regexes are probably applicable to most Hugo/blogdown setups.
rgx_replace <- '(content\\/post\\/)(.*)(\\/)(.*)([.]png$)'
rgx_title <- '^title[:]\\s+'
rgx_date <- 'date[:]\\s+'

# This regex is particular to the way that I name and save my ggplots.
rgx_viz <- '(^[!][\\[][\\]].*)(viz.*png)(.*$)'

# Define a helper function for a common idiom that we will implement for extracting the ines of markdown that we 
# want---those containing the title, data, and visualization---and  trimming them just to the text that we want
# (i.e. removing "title:" and "date:" preceding the title and date in the YAML/TOML header, and
# removing the "![]" preceding an image).
str_pluck <- function(x, pattern, replacement = '') {
  x %>% 
    str_subset(pattern) %>% 
    str_replace_all(pattern = pattern, replacement = replacement) %>% 
    str_trim()
}
str_pluck_title <- purrr::partial(str_pluck, pattern = rgx_title)
str_pluck_date <- purrr::partial(str_pluck, pattern = rgx_date)
str_pluck_viz <- purrr::partial(str_pluck, pattern = rgx_viz, replacement = '\\2')

# Extract the title, date, and visualizations from each post.
# Note that there should be only one title and date pers post, but there are likely more than one visualization per post.
paths_post <-
  paths_post_raw %>% 
  as.character() %>% 
  tibble(path_post = .) %>% 
  mutate(
    lines = path_post %>% purrr::map(read_lines)
  ) %>% 
  mutate_at(vars(path_post), ~str_remove_all(., 'content|\\/index[.]md')) %>% 
  mutate_at(
    vars(lines),
    list(
      title = ~purrr::map_chr(., str_pluck_title),
      date = ~purrr::map_chr(., str_pluck_date) %>% lubridate::ymd(),
      viz = ~purrr::map(., str_pluck_viz)
    )
  ) %>% 
  # viz` is a list item (because there may be more than one per post), so we need to `unnest()` it to return a "tidy" data frame.
  unnest(viz) %>% 
  select(date, viz, title, path_post)
paths_post

## # A tibble: 71 x 4
##    date       viz             title                   path_post            
##                                                       
##  1 2018-05-20 viz_map_bycomp~ An Analysis of Texas H~ /post/analysis-texas~
##  2 2018-05-20 viz_map_bycomp~ An Analysis of Texas H~ /post/analysis-texas~
##  3 2018-05-20 viz_n_bycomplv~ An Analysis of Texas H~ /post/analysis-texas~
##  4 2018-05-20 viz_n_bycomp-1~ An Analysis of Texas H~ /post/analysis-texas~
##  5 2018-05-20 viz_n_bycompco~ An Analysis of Texas H~ /post/analysis-texas~
##  6 2018-05-20 viz_n_bycompco~ An Analysis of Texas H~ /post/analysis-texas~
##  7 2018-05-20 viz_persons_st~ An Analysis of Texas H~ /post/analysis-texas~
##  8 2018-05-20 viz_persons_st~ An Analysis of Texas H~ /post/analysis-texas~
##  9 2018-05-20 viz_persons_st~ An Analysis of Texas H~ /post/analysis-texas~
## 10 2018-05-20 viz_persons_st~ An Analysis of Texas H~ /post/analysis-texas~
## # ... with 61 more rows

# Create the markdown lines for images (visualizations) for our gallery markdown output.
paths_post_md <-
  paths_post %>% 
  mutate(
    label_md = sprintf('![%s](%s/%s)', viz, path_post, viz)
  ) %>% 
  select(title, date, path_post, label_md)
paths_post_md

## # A tibble: 71 x 4
##    title               date       path_post           label_md             
##                                                       
##  1 An Analysis of Tex~ 2018-05-20 /post/analysis-tex~ ![viz_map_bycomplvl_~
##  2 An Analysis of Tex~ 2018-05-20 /post/analysis-tex~ ![viz_map_bycomplvl_~
##  3 An Analysis of Tex~ 2018-05-20 /post/analysis-tex~ ![viz_n_bycomplvl-1.~
##  4 An Analysis of Tex~ 2018-05-20 /post/analysis-tex~ ![viz_n_bycomp-1.png~
##  5 An Analysis of Tex~ 2018-05-20 /post/analysis-tex~ ![viz_n_bycompcomplv~
##  6 An Analysis of Tex~ 2018-05-20 /post/analysis-tex~ ![viz_n_bycompcomplv~
##  7 An Analysis of Tex~ 2018-05-20 /post/analysis-tex~ ![viz_persons_stats_~
##  8 An Analysis of Tex~ 2018-05-20 /post/analysis-tex~ ![viz_persons_stats_~
##  9 An Analysis of Tex~ 2018-05-20 /post/analysis-tex~ ![viz_persons_stats_~
## 10 An Analysis of Tex~ 2018-05-20 /post/analysis-tex~ ![viz_persons_stats_~
## # ... with 61 more rows

# Create the "main" data frame with the titles and dates in columns alongside the image column.
# In "tidy data" terminology, images are the "observations" *and `title` and `date` are separate variables).
content_gallery_raw <-
  paths_post_md %>%
  group_by(title, date, path_post) %>%
  # Add a "placeholder" line for the title of the post.
  do(add_row(., .before = 0)) %>% 
  ungroup() %>% 
  # In the first case, create the H2 markkdown heading line for the name of the post.
  # In the second case, use the image markdown line created above.
  mutate_at(
    vars(label_md),
    ~case_when(
      is.na(.) ~ sprintf('## %s, [%s](%s)', dplyr::lead(date), dplyr::lead(title), dplyr::lead(path_post)),
      TRUE ~ .
    )
  ) %>% 
  # Impute the `title` and `date` values to go with the image values.
  fill(title, .direction = 'up') %>% 
  fill(date, .direction = 'up') %>% 
  arrange(date) %>% 
  # Number the posts in order of descending date.
  mutate(idx_intragrp = dense_rank(sprintf('%s, %s', date, title))) %>%
  group_by(title, date) %>% 
  # Number the images within each post. (This isn't completely necessary. It's only used for sorting.)
  mutate(idx_intergrp = row_number()) %>% 
  ungroup() %>% 
  select(idx_intragrp, idx_intergrp, date, label_md) %>% 
  arrange(desc(idx_intragrp), idx_intergrp)
content_gallery_raw

## # A tibble: 90 x 4
##    idx_intragrp idx_intergrp date       label_md                           
##                                                       
##  1           19            1 2019-06-29 ## 2019-06-29, [Text Parsing and T~
##  2           19            2 2019-06-29 ![viz_toc_n_1yr_tree.png](/post/te~
##  3           19            3 2019-06-29 ![viz_content_section_n.png](/post~
##  4           19            4 2019-06-29 ![viz_toc_content_n1.png](/post/te~
##  5           19            5 2019-06-29 ![viz_sents_section_n.png](/post/t~
##  6           19            6 2019-06-29 ![viz_sents_section_n_yr.png](/pos~
##  7           19            7 2019-06-29 ![viz_sents_section_sim.png](/post~
##  8           19            8 2019-06-29 ![viz_words_section_tfidf.png](/po~
##  9           19            9 2019-06-29 ![viz_words_tfidf.png](/post/text-~
## 10           18            1 2019-01-27 ## 2019-01-27, [Summarizing rstudi~
## # ... with 80 more rows

# Create the final markdown output.
content_gallery <-
  content_gallery_raw %>% 
  select(label_md) %>% 
  mutate(idx = row_number()) %>% 
  # Add a blank line between the end of one section's last image
  # and the next sections H2 header.
  group_by(idx) %>% 
  do(add_row(., label_md = '', .before = 0)) %>% 
  ungroup()
content_gallery

## # A tibble: 180 x 2
##    label_md                                                             idx
##                                                                  
##  1 ""                                                                    NA
##  2 ## 2019-06-29, [Text Parsing and Text Analysis of a Periodic Repo~     1
##  3 ""                                                                    NA
##  4 ![viz_toc_n_1yr_tree.png](/post/text-parsing-analysis-periodic-re~     2
##  5 ""                                                                    NA
##  6 ![viz_content_section_n.png](/post/text-parsing-analysis-periodic~     3
##  7 ""                                                                    NA
##  8 ![viz_toc_content_n1.png](/post/text-parsing-analysis-periodic-re~     4
##  9 ""                                                                    NA
## 10 ![viz_sents_section_n.png](/post/text-parsing-analysis-periodic-r~     5
## # ... with 170 more rows

content_copypaste <- content_gallery %>% pull(label_md)

# Copy paste this to the markdown file for the gallery page.
# It's probably possible to do this a bit more programmatically (i.e. without
# "manually" copying into a markdown file, but oh well
clipr::write_clip(content_copypaste)

Continue Reading…

Collapse

Read More

highlightjsVersion = "9.12.0"
highlightjsCDN = "//cdnjs.cloudflare.com/ajax/libs"
highlightjsLang = ["r", "yaml"]
highlightjsTheme = "github"

Continue Reading…

Collapse

Read More

Very often, seed scientists need to compare the germination behaviour of different seed populations, e.g., different plant species, or one single plant species submitted to different temperatures, light conditions, priming treatments and so on. How should such a comparison be performed?

Let’s take a practical approach and start from an appropriate example: a few years ago, some collegues studied the germination behaviour for seeds of a plant species (Verbascum arcturus, BTW…), in different conditions. In detail, they considered the factorial combination of two storage periods (LONG and SHORT storage) and two temperature regimes (FIX: constant daily temperature of 20°C; ALT: alternating daily temperature regime, with 25°C during daytime and 15°C during night time, with a 12:12h photoperiod). If you are a seed scientist and are interested in this experiment, you’ll find detail in Catara et al. (2016).

If you are not a seed scientist you may wonder why my colleagues made such an assay; well, there is evidence that, for some plant species, the germination ability improves over time after seed maturation. Therefore, if we take seeds and store them for a different period of time, there might be an effect on their germination traits. Likewise, there is also evidence that some seeds may not germinate if they are not submitted to daily temperature fluctuations. These mechanisms are very interesting, as they permit to the seed to recognise that the environmental conditions are favourable for seedling survival.My colleagues wanted to discover whether this was the case for Verbascum.

Let’s go back to our assay: the experimental design consisted of four combinations (LONG-FIX, LONG-ALT, SHORT-FIX and SHORT-ALT) and four replicates for each combination. One replicate consisted of a Petri dish, that is a small plastic box containing humid blotting paper, with 25 seeds of Verbascum. In all, there were 16 Petri dishes, put in climatic chambers with the appropriate conditions. During the assay, my collegues made daily inspections: germinated seeds were counted and removed from the dishes. Inspections were made for 15 days, until no more germinations could be observed.

The dataset is available from a gitHub repository: let’s load it and have a look.

dataset <- read.csv("https://raw.githubusercontent.com/OnofriAndreaPG/agroBioData/master/TempStorage.csv", header = T, check.names = F)
head(dataset)
##   Dish Storage Temp 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
## 1    1     Low  Fix 0 0 0 0 0 0 0 0 3  4  6  0  1  0  3
## 2    2     Low  Fix 0 0 0 0 1 0 0 0 2  7  2  3  0  5  1
## 3    3     Low  Fix 0 0 0 0 1 0 0 1 3  5  2  4  0  1  3
## 4    4     Low  Fix 0 0 0 0 1 0 3 0 0  3  1  1  0  4  4
## 5    5    High  Fix 0 0 0 0 0 0 0 0 1  2  5  4  2  3  0
## 6    6    High  Fix 0 0 0 0 0 0 0 0 2  2  7  8  1  2  1

We have one row per Petri dish; the first three columns show, respectively, dish number, storage and temperature conditions. The next 15 columns represent the inspection times (from 1 to 15) and contain the counts of germinated seeds. The research question is:

Is germination behaviour affected by storage and temperature conditions?

library(drcSeedGerm)
datasetR <- makeDrm(dataset[,4:18], dataset[,2:3], rep(25, 16), 1:15)
head(datasetR, 16)
##      Storage Temp Dish timeBef timeAf count nCum propCum
## 1        Low  Fix    1       0      1     0    0    0.00
## 1.1      Low  Fix    1       1      2     0    0    0.00
## 1.2      Low  Fix    1       2      3     0    0    0.00
## 1.3      Low  Fix    1       3      4     0    0    0.00
## 1.4      Low  Fix    1       4      5     0    0    0.00
## 1.5      Low  Fix    1       5      6     0    0    0.00
## 1.6      Low  Fix    1       6      7     0    0    0.00
## 1.7      Low  Fix    1       7      8     0    0    0.00
## 1.8      Low  Fix    1       8      9     3    3    0.12
## 1.9      Low  Fix    1       9     10     4    7    0.28
## 1.10     Low  Fix    1      10     11     6   13    0.52
## 1.11     Low  Fix    1      11     12     0   13    0.52
## 1.12     Low  Fix    1      12     13     1   14    0.56
## 1.13     Low  Fix    1      13     14     0   14    0.56
## 1.14     Low  Fix    1      14     15     3   17    0.68
## 1.15     Low  Fix    1      15    Inf     8   NA      NA

modPre <- drm(count ~ timeBef + timeAf, fct = LL.3(), 
              data = datasetR, 
              type = "event", subset = c(Dish == 1))
plot(modPre, log = "", xlab = "Time", 
     ylab = "Proportion of germinated seeds",
     xlim = c(0, 15))

mod1 <- drm(count ~ timeBef + timeAf, fct = LL.3(),
            data = datasetR, type = "event", 
            curveid = Temp:Storage)
plot(mod1, log = "", legendPos = c(2, 1))

modNull <- drm(count ~ timeBef + timeAf, fct = LL.3(),
               data = datasetR, 
               type = "event")
anova(mod1, modNull, test = "Chisq")
## 
## 1st model
##  fct:      LL.3()
##  pmodels: Temp:Storage (for all parameters)
## 2nd model
##  fct:      LL.3()
##  pmodels: 1 (for all parameters)
## ANOVA-like table
## 
##           ModelDf  Loglik Df LR value p value
## 1st model     244 -753.54                    
## 2nd model     253 -854.93  9   202.77       0

mod2 <- drm(count ~ timeBef + timeAf, fct = LL.3(), 
            data = datasetR, type = "event", 
            curveid = Storage)
anova(mod1, mod2, test = "Chisq")
## 
## 1st model
##  fct:      LL.3()
##  pmodels: Temp:Storage (for all parameters)
## 2nd model
##  fct:      LL.3()
##  pmodels: Storage (for all parameters)
## ANOVA-like table
## 
##           ModelDf  Loglik Df LR value p value
## 1st model     244 -753.54                    
## 2nd model     250 -797.26  6   87.436       0

mod3 <- drm(count ~ timeBef + timeAf, fct = LL.3(), 
            data = datasetR, type = "event", 
            curveid = Temp)
anova(mod1, mod3, test = "Chisq")
## 
## 1st model
##  fct:      LL.3()
##  pmodels: Temp:Storage (for all parameters)
## 2nd model
##  fct:      LL.3()
##  pmodels: Temp (for all parameters)
## ANOVA-like table
## 
##           ModelDf  Loglik Df LR value p value
## 1st model     244 -753.54                    
## 2nd model     250 -849.48  6   191.87       0

AIC(mod1, mod2, mod3, modNull)
##          df      AIC
## mod1    244 1995.088
## mod2    250 2094.524
## mod3    250 2198.961
## modNull 253 2215.862

Continue Reading…

Collapse

Read More

Facebook owes its resilience to savvy acquisitions and tolerant regulators

Continue Reading…

Collapse

Read More

A couple people pointed me to this news story, “A rise in premature births among Latina women may be linked to Trump’s election, study says,” and the associated JAMA article, which begins:

Question Did preterm births increase among Latina women who were pregnant during the 2016 US presidential election?

Findings This population-based study used an interrupted time series design to assess 32.9 million live births and found that the number of preterm births among Latina women increased above expected levels after the election.

Meaning The 2016 presidential election may have been associated with adverse health outcomes of Latina women and their newborns.

Hmmm, the research article says “may have been associated” but then ups that to “appears to have been associated.”

On one hand, I find it admirable that JAMA will publish a paper with such an uncertain conclusion. On the other hand, the conclusions got stronger once they made their way into news reports. In the above-linked article, “may have been associated” becomes “an association was found” and then “We think there are very few alternative explanations for these results.”

There’s also a selection issue. It’s fine to report maybes, but then why this particular maybe? There are lots and lots of associations that may be happening, right?

Let’s look at the data

In any case, they did an interrupted time series analysis, so let’s see the time series:

I don’t think the paper’s claim, “In the 9-month period beginning with November 2016, an additional 1342 male (95% CI, 795-1889) and 995 female (95% CI, 554-1436) preterm births to Latina women were found above the expected number of preterm births had the election not occurred,” is at all well supported by these data. But you can make your own judgement here.

Also, I’m surprised they are analyzing raw numbers of pre-term births rather than rates.

In general

Medical journals sometimes seem to show poor judgment when it comes to stories that agree with their political prejudices. See for example here and here.

Look. Don’t get me wrong. This topic is important. We’d like to minimize preterm births, and graphs such as shown above (ideally using rates, not counts, I think) should be a key part of a monitoring system that will allow us to notice problems. It should be possible to look at such time series without pulling out one factor and wrapping this sort of story around it. I think this is a problem with scientific publication, that journals and the news media want to publish big splashy claims.

Continue Reading…

Collapse

Read More

Flash floods seem to be increasing in many areas. This post will show how to download local USGS flow and precipitation data and generate a 3-panel chart of flow, gage height and precipitation.

There was a tragic flash flood incident in Berks County Pennsylvania on July 11-12, 2019 that caused the death of 3 people, including a pregnant woman and her young son. This newspaper article (link) provides some details on this tragic event.

The timing of this flash flooding was interesting to me because the woman and her son were trapped in their car by a small tributary to the Schuylkill River at 4:30 PM on Thursday, July 11 and flooding in Philadelphia started about 1:30 Am on Friday, July 12th. The tragic drowning occurred on the Manatawny Creek , about 35 miles Northwest of Philadelphia.

I plan to write a series of posts to document the rainfall patterns and timing of the rainfall and subsequent flooding.

This initial post will present show the flow hydrograph, cumulative precipitation and gage height at for the USGS’s Schuylkill river gaging station #01474500 at Fairmount Dam.

Follow-up posts will review the upstream USGS data as well as national weather service rainfall data.

Here is the 3 panel chart showing the hydrograph, precipitation data and gage height data for the period July 9 – 16, 2019.

Here’s the R script for analyzing and plotting csv file downloaded from USGS site.

#################################################################################################################

## get source data file link
link <- "C:\\Users\\Kelly O'Day\\Desktop\\Schuylkill_flood\\CSV_files\\Schuylkil_Fairmont_Dam_7_days_july_2019.csv"
#(link <- choose.files())
df <- read.csv(link, as.is=T)
df$dt <- mdy_hm(df$datetime)
dt_min <- df$dt[1]
dt_max <- df$dt[nrow(df)]

peak_flow <- max(df$cfs, na.rm=T)
peak_flow_comma <- format(peak_flow, big.mark=',')
peak_dt <- df$dt[which(df$cfs == peak_flow)]
from_to <- paste(month(dt_min),"/",day(dt_min), " to ", month(dt_max),"/",day(dt_max),"/",year(dt_max),sep="")

# Create 3 panel charts for flow, precipitation and gage height
par(mfrow = c(3,1),ps=10, pty="m",xaxs="r",yaxs="r")
par(las=1,oma=c(2.5,2,3,1), mar=c(2,5,3,0))
par(mgp=c(4,1,0)) # Number of margin lines for title, axis labels and axis line

## Flow chart
plot_title <- paste("Flow - cfs")
plot(df$dt, df$cfs, type="l", xlab="", ylab = "Flow -cfs",xlim=c(dt_min,dt_max), yaxt="n",
main =plot_title)
axis(side=2, at=axTicks(2),
labels=formatC(axTicks(2), format="d", big.mark=','))

points(peak_dt, peak_flow, pch=16, col="red")
spacer <- 4*60*60
note <- paste("Peak @ ", peak_flow_comma, " cfs \n", peak_dt)
abline(h = seq(10000,50000,10000), col = "grey")
abline(v= seq(dt_min,dt_max,by="day"), col="grey")
text(peak_dt+ spacer,peak_flow * 0.975, note, adj=0, cex=1)
################################################
# Sheet title annotation
mtext("Flood Conditions @ Fairmount Dam (July 9-16, 2019)", side = 3, line = 3, adj = 0.5, cex = 1.2)

##### precipitation data analyis n& Chart
df$cum_precip <- cumsum(df$inches)
tot_precip <- df$cum_precip[nrow(df)]
precip_st_row <- length(subset(df$cum_precip, df$cum_precip == 0))
precip_st_dt <- df$dt[precip_st_row]

precip_note <- paste0("Total Precip - ",tot_precip, " inches")
precip_subset <- subset(df, df$cum_precip == tot_precip)
precip_end_dt <- mdy_hm(precip_subset[1,1])

#precip_end_dt_t <- mdy_hm(precip_end_dt)
plot(df$dt, df$cum_precip, type="l", xlab="", ylab = "Precipitation -inches",xlim=c(dt_min,dt_max),
main = "Cummulative Precipitation - Inches")
points(precip_st_dt,0, pch=12, col = "blue")
points(precip_end_dt,tot_precip, pch=12, col="blue")
abline(v= seq(dt_min,dt_max,by="day"), col="grey")
text(dt_min,tot_precip, precip_note, adj=0)
precip_st_note <- paste0(" Precipitation Starts @ ",precip_st_dt)
dur <- precip_end_dt - precip_st_dt

precip_end_note <- paste0(" Precipitation ends @ ",precip_end_dt,"\n ",dur, " hours")
text(precip_end_dt,tot_precip * 0.9, precip_end_note, adj=0, cex = 1)
text(precip_st_dt, 0, precip_st_note, adj=0, cex = 1)

#### gage height chart
gage_act_df <- subset(df, df$gage >= 10)
gage_act_dt <- gage_act_df$dt[1]
gage_act_note <- paste0("Gage Action Level @ \n",gage_act_dt)
plot(df$dt, df$gage, type="l", xlab="", ylab = "Gage Height - Ft",xlim=c(dt_min,dt_max),
main ="Gage height - ft")
abline(h = 10, col="brown")
abline(h=11, col = "red", lwd =1)
abline(v= seq(dt_min,dt_max,by="day"), col="grey")
points(gage_act_dt,gage_act_df[1,3],pch=11, col = "black")
text(gage_act_dt - 1*4*3600, 10,gage_act_note, adj=1)
min_gage <- min(df$gage, na.rm=T)
max_gage <- max(df$gage, na.rm=T)
delta_gage <- max_gage - min_gage
gage_note <- paste("Max Gage @ ", max_gage, " ft\nGage Increase ", delta_gage, " ft")
text(dt_min, max_gage*0.97, gage_note, adj=0, cex=1)

flood_act_note <- "Flood Action stage @ 10-ft"
flood_stage_note <-"Flood stage @ 11-ft"
text(dt_max-60*60,10.25, flood_act_note, adj = 1, cex=0.95)
text(dt_max - 1*1*60*60,11.25, flood_stage_note, adj = 1, cex=0.95)

##########################################
# Sheet annotation - Footer Notes
mtext("K O'Day - 7/19/19", side = 1, line = 3, adj = 0, cex = 0.9)
mtext("Data Source: USGS: https://waterdata.usgs.gov/nwis/inventory/?site_no=01474500", side=1, line = 2, adj=0, cex = 0.9)

# png(file="C:\\Users\\Kelly O'Day\\Desktop\\Schuylkill_flood\\myplot.png", bg="white")

dev.off()

Continue Reading…

Collapse

Read More

The keynote presentations from last week's useR!2019 conference in Toulouse are now available for everyone to view on YouTube. (The regular talks were also recorded and video should follow soon, and slides for most talks are available for download now at the conference website.) Here are links to the videos, indexed to the start of each presentation:

• Shiny's Holy Grail: Interactivity with reproducibility.  Joe Cheng, Chief Technology Officer at RStudio 
• "AI for Good" in the R and Python ecosystems. Julien Cornebise, Director of Research at Element AI
• Tools for Model-Based Clustering in R. Bettina Grün, Professor at Johannes Kepler Universität Linz
• A missing value tour in R. Julie Josse, Professor at École Polytechnique
• How Bioconductor advances science while contributing to the R language and community. Martin Morgan, Professor at Roswell Park Comprehensive Cancer Center and Bioconductor
• R for better science in less time. Julia Stewart Lowndes, Mozilla Fellow and Marine Data Scientist at NCEAS

All of the presentations are excellent, but if I had to choose one to watch first, it would be Julia Stewart Lowndes' presentation, which is an inspiring example of how R has enabled marine researchers to collaborate and learn from data (like a transponder-equipped squid!). 

The videos have been made available thanks to sponsorship by the R Consortium. If you're not familiar with the R Consortium, you can learn more in the short presentation below from Joe Rickert:

The R Consortium is funded by its member organizations, so if you'd like to see more of the above, consider asking your company to become a member.

YouTube (R Consortium): 2019 Keynotes

Continue Reading…

Collapse

Read More

Continue Reading…

Collapse

Read More

Journal Mining, API Use, Better Conversation, and Apollo 11 Source

1. 73 Million Journal Articles for Text Mining (BoingBoing) -- The JNU Data Depot is a joint project between rogue archivist Carl Malamud, bioinformatician Andrew Lynn, and a research team from New Delhi's Jawaharlal Nehru University: together, they have assembled 73 million journal articles from 1847 to the present day and put them into an airgapped respository that they're offering to noncommercial third parties who want to perform textual analysis on them to "pull out insights without actually reading the text."
2. How Developers Use API Documentation: An Observation Study (ACM) -- participants totally mapped to opportunistic (risk-taking, paste-then-adapt, change-without-checking) developers and systematic (start with clean code, read the docs, learn before coding) developers.
3. Talk -- An open source commenting platform focused on better conversation.
4. Apollo 11 -- Original Apollo 11 Guidance Computer (AGC) source code for the command and lunar modules.

Continue reading Four short links: 19 July 2019.

Continue Reading…

Collapse

Read More

Monash is seeking 2 x Lecturer / Senior Lecturer within the space of Digital Health - Image Analytics. Digital Health is a fascinating cross-university, cross-faculty, multidisciplinary space with enormous practical real-world and societal benefits that draws from and contributes to a multiplicity of research areas.

Continue Reading…

Collapse

Read More

Seeking a customer-focused, passionate and driven Data Science Engineer with experience in building analytic tools and solutions.

Continue Reading…

Collapse

Read More

All you need to know about data pre-processing, and how to build and optimize a regression model using Backward Elimination method in Python.

Continue Reading…

Collapse

Read More