Review: DARK EMU


Dark Emuby Bruce Pascoe is a powerful, compelling work that achieves its dual aims: showing just how complex and well-developed the civilisation managing the Australian continent was prior to European contact and subsequent colonisation, and the lengths that have been gone to erase this vital pre-history from the collective minds of the current occupying Australian civilisation.

It brings…

View On WordPress

There’s a family of interesting related arguments: “You are not enough people” (Vonnegut theory of marital conflict) “Reality…

IFTTT, Twitter, vgr


Inexplicably I’ve accepted the challenge by @akrishnan23 to post the covers of 7 books that I love/recommend: no explanations,…

IFTTT, Twitter, changeist


It takes a bot to know one?


A couple of weeks ago, I wrote about GPT-2, a text-generating algorithm whose huge size and long-term analysis abilities mean that it can generate text with an impressive degree of coherence. So impressive, in fact, that its programmers at OpenAI have only released a mini version of the model for now, worried that people may abuse the full-size model’s easy-to-generate, almost-plausibly-human text. 

(below: some text generated by mini-GPT-2, in response to the prompt in italics)

This was a fantastic recipe for chocolate cake with raspberry sauce! I only made a couple of changes to the recipe. First, Iadded vanilla candles instead of meringues for a more mild and exotic fragrance. Once again, I only used 1 tsp of vanilla syrup for clarity. Second, the chocolate cake whipped cream was tempered by an additional 1 tsp of canola oil. The regular vegan whipped cream is soothing and makes it pleasing to the hungry healthiest person I know!

In the meantime, as OpenAI had hoped, people are working on ways to automatically detect GPT-2′s text. Using a bot to detect another bot is a strategy that can work pretty well for detecting fake logins, video, or audio. And now, a group from MIT-IBM Watson AI lab and Harvard NLP has come up with a way of detecting fake text, using GPT-2 itself as part of the detection system.

The idea is fairly simple: GPT-2 is better at predicting what a bot will write than what a human will write. So if GPT-2 is great at predicting the next word in a bit of text, that text was probably written by an algorithm - maybe even by GPT-2 itself.

There’s a web demo that they’re calling Giant Language model Test Room (GLTR), so naturally I decided to play with it.

First, here’s some genuine text generated by GPT-2 (the full-size model, thanks to the OpenAI team being kind enough to send me a sample). Green words are ones that GLTR thought were very predictable, yellow and red words are less predictable, and purple words are ones the algorithm definitely didn’t see coming. There are a couple of mild surprises here, but mostly the AI knew what would be generated. Seeing all this green, you’d know this text is probably AI-generated.

HERMIONE: So, you told him the truth?
Snape: Yes.
HARRY: Is it going to destroy him? You want him to be able to see the truth.
Snape: [turning to her] Hermione, I-I-I'm not looking for acceptance.
HARRY: [smiling] No, it's-it's good it doesn't need to be.
Snape: I understand.
	[A snake appears and Snape puts it on his head and it appears to do the talking. 	It says 'I forgive you.']
HARRY: You can't go back if you don't forgive.
Snape: [sighing] Hermione.
HARRY: Okay, listen.
Snape: I want to apologize to you for getting angry and upset over this.
HARRY: It's not your fault.
HARRY: That's not what I meant to imply.
	[Another snake appears then it says 'And I forgive you.']
HERMIONE: And I forgive you.
Snape: Yes.

Here, on the other hand, is how GLTR analyzed some human-written text, the opening paragraph of the Murderbot diaries. There’s a LOT more purple and red. It found this human writer to be more unpredictable.

I could have become a mass murderer after I hacked my governor module, but then I realized I could access the combined feed of entertainment channels carried on the company satellites. It had been well over 35,000 hours or so since then, with still not much murdering, but probably, I don’t know, a little under 35,000 hours of movies, serials, books, plays, and music consumed. As a heartless killing machine, I was a terrible failure.

But can GLTR detect text generated by another AI, not just text that GPT-2 generates? It turns out it depends. Here’s text generated by another AI, the Washington Post’s Heliograf algorithm that writes up local sports and election results into simple but readable articles. Sure enough, GLTR found Heliograf’s articles to be pretty predictable. Maybe GPT-2 had even read a lot of Heliograf articles during training.


However, here’s what it did with a review of Avengers: Infinity War that I generated using an algorithm Facebook trained on Amazon reviews. It’s not an entirely plausible review, but to GLTR it looks a lot more like the human-written text than the AI-generated text. Plenty of human-written text scores in this range.

The Avengers: Infinity War is a movie that should be viewed on its own terms, and not a tell-all about The Hulk.  I have always loved the guys that played Michael Myers, and of all the others like Angel and Griffin, Kim back to Bullwinkle, and Edward James Olmos as the Lion. Special mention must go to the performances of Robert De Niro and Anthony Hopkins.  Just as I would like to see David Cronenberg in a better role, he is a treat the way he is as Gimli.Also there is the evil genius Bugs Bunny and the amazing car chase scene that has been hailed as THE Greatest Tank Trio of All Time ever (or at least the last one).  With Gary Oldman and Robert Young on the run and almost immediate next day in the parking lot to be his lover, he tries to escape in a failed attempt at a new dream.  It was a fantastic movie, full of monsters and beasts, and makes the animated movies seem so much more real.

And here’s how GLTR rated another Amazon review by that same algorithm. A human might find this review to be a bit suspect, but, again, the AI didn’t score this as bot-written text.

The Harry Potter File, from which the previous one was based (which means it has a standard size liner) weighs a ton and this one is huge! I will definitely put it on every toaster I have in the kitchen since, it is that good.This is one of the best comedy movies ever made. It is definitely my favorite movie of all time. I would recommend this to ANYONE!

What about an AI that’s really, really bad at generating text? How does that rate? Here’s some output from a neural net I trained to generate Dungeons and Dragons biographies. Whatever GLTR was expecting, it wasn’t fuse efforts and grass tricks.

instead was a drow, costumed was toosingly power they are curious as his great embercrumb, a fellow knight of the area of the son, and the young girl is the agents guild, as soon as she received astering the grass tricks that he could ask to serve his words away and he has a disaster of the spire, but he was super connie couldn't be resigned to the church, really with the fuse effort to fit the world, tempting into the church of the moment of the son of the gods, there was what i can contrive that she was born into his own life, pollaning the bandit in the land. the ship, i decided to fight with the streets. he met the ship without a new priest of pelor like a particularly bad criters but was assigned as he was sat the social shape and his desire over the river and a few ways that had been seriously into the fey priest. abaewin was never taken in the world. he had told me this was lost for it, for reason, and i cant know what was something good clear, but she had attack them 15, they were divided by a visators above the village, but he went since i was so that he stayed. but one day, she grew up from studying a small lion.

But I generated that biography with the creativity setting turned up high, so my algorithm was TRYING to be unpredictable. What if I turned the D&D bio generator’s creativity setting very low, so it tries to be predictable instead? Would that make it easier for GLTR to detect? Only slightly. It still looks like unpredictable human-written text to GLTR.

he is a successful adventurers of the city and the lady of the undead who would be able to use his own and a few days in the city of the city of bandits. he was a child to be a deadly in the world and the goddess of the temple of the city of waterdeep. he was a child for a few hours and the incident of the order of the city and a few years of research. she was a child in a small village and was invited to be a good deal in the world and in the world and the other children of the tribe and the elven village and the young man was exiled in the world. he was a child to the forest to the local tavern and a human bard, a human bard in a small town of his family and the other two years of a demon in the world.

GLTR is still pretty good at detecting text that GPT-2 generates - after all, it’s using GPT-2 itself to do the predictions. So, it’ll be a useful defense against GPT-2 generated spam.

But, if you want to build an AI that can sneak its text past a GPT-2 based detector, try building one that generates laughably incoherent text. Apparently, to GPT-2, that sounds all too human.

For more laughably incoherent text, I trained a neural net on the complete text of Black Beauty, and generated a long rambling paragraph about being a Good Horse. To read it, and GLTR’s verdict, enter your email here and I’ll send it to you.

Towards a general theory of “adversarial examples,” the bizarre, hallucinatory motes in machine learning’s all-seeing eye


For several years, I’ve been covering the bizarre phenomenon of “adversarial examples (AKA “adversarial preturbations”), these being often tiny changes to data than can cause machine-learning classifiers to totally misfire: imperceptible squeaks that make speech-to-text systems hallucinate phantom voices; or tiny shifts to a 3D image of a helicopter that makes image-classifiers hallucinate a rifle  

A friend of mine who is a very senior cryptographer of longstanding esteem in the field recently changed roles to managing information security for one of the leading machine learning companies: he told me that he thought that it may be that all machine-learning models have lurking adversarial examples and it might be impossible to eliminate these, meaning that any use of machine learning where the owners of the system are trying to do something that someone else wants to prevent might never be secure enough for use in the field – that is, we may never be able to make a self-driving car that can’t be fooled into mistaking a STOP sign for a go-faster sign.

What’s more there are tons of use-cases that seem non-adversarial at first blush, but which have potential adversarial implications further down the line: think of how the machine-learning classifier that reliably diagnoses skin cancer might be fooled by an unethical doctor who wants to generate more billings; or nerfed down by an insurer that wants to avoid paying claims.

My MIT Media Lab colleague Joi Ito (previously) has teamed up with Harvard’s Jonathan Zittrain (previously to teach a course on Applied Ethical and Governance Challenges in AI, and in reading the syllabus, I came across Motivating the Rules of the Game for Adversarial Example Research, a 2018 paper by a team of Princeton and Google researchers, which attempts to formulae a kind of unified framework for talking about and evaluating adversarial examples.

The authors propose a taxonomy of attacks, based on whether the attackers are using “white box” or “black box” approaches to the model (that is, whether they are allowed to know how the model works), whether their tampering has to be imperceptible to humans (think of the stop-sign attack – it works best if a human can’t see that the stop sign has been altered), and other factors.

It’s a fascinating paper that tries to make sense of the to-date scattershot adversarial example research. It may be that my cryptographer friend is right about the inevitability of adversarial examples, but this analytical framework goes a long way to helping us understand where the risks are and which defenses can or can’t work.

If this kind of thing interests you, you can check out the work that MIT Media Lab students are doing with Labsix, a student-only, no-faculty research group that studies adversarial examples.

Correcting the Record on the First Emoji Set


The Emojipedia blog has an important update in emoji history news!

Until now, Japanese phone carrier Docomo has most often been widely credited as the originator of what we know as emoji today. It turns out, that might not be the case, and today we are correcting the record.

SoftBank, the carrier that partnered with Apple to bring the iPhone to Japan in 2008, released a phone with support for 90 distinct emoji characters in 1997. For the first time, these are now available on Emojipedia.

The 90 emojis from SoftBank in 1997 predate the set of 176 emojis released by Docomo in 1999, which until now have most commonly been cited (including by Emojipedia) as being the first.

Not only was the 1997 SoftBank emoji set released earlier than the first known date of the Docomo emoji set (in “1998 or 1999”), one of the most iconic emoji characters now encoded as 💩 U+1F4A9 PILE OF POO in the Unicode Standard, originated in this release.

Unless or until we find evidence that Docomo had an emoji set available prior to this release, we hereby issue a correction that the original emoji set is from SoftBank in Japan in 1997, with designer/s unknown.

Read the whole post for more emoji history.

Correcting the Record on the First Emoji Set

There exists a series of underground bunkers with no entrances or exits of any kind scattered across North America, each of…

IFTTT, Twitter, ThePatanoiac


ONLY THE BEGINNING OF ANOTHER STRANGENESS - what happens when AI meets the alien consciousnesses that already live amongst us? I…

IFTTT, Twitter, jamesbridle


I’m in the non-apocalyptic camp of climate change believers. I don’t believe even the worst case will make the world…

IFTTT, Twitter, vgr


I am SO thrilled to announce Atmospheric Memory, an epic new production by Rafael Lozano Hemmer (@errafael) curated by…

IFTTT, Twitter, Macroscopist


AI-created fine art could enliven galleries with visual aesthetics that humans couldn’t foresee, or it could become a…

IFTTT, Twitter, ibogost


Actually, lemme try mapping the 8 metaphors Mechanistic: GTD Brain: BASB/PKM Organism: Blitzkrieg model Culture: Improv theater…

IFTTT, Twitter, vgr


‘The documents were written on hundreds of strips of bamboo, about the size of chopsticks, that seemed to date from 2,500 years…

IFTTT, Twitter, justinpickard


Something I’ve been meaning to say about The Tragedy of the Commons. Bear with me for a small thread on why our embrace of…

IFTTT, Twitter, mmildenberger


Went to a talk last week by someone in the Texas Bureau of Economic Geology & it’s safe to say that many in the oil & gas…

IFTTT, Twitter, PeterBrannen1


A summary of Octavia E. Butler’s advice for writers: 1. “Read omnivorously…” 2. “Forget about talent.” 3. "Write, every day,…

IFTTT, Twitter, tamaranopper


’[D]rugs do not impart wisdom…any more than the microscope alone gives knowledge.They provide the raw materials of wisdom&are…

IFTTT, Twitter, PeterSjostedtH


Crop Yield Prediction Gold. Predictive systems are being heavily invested in by DARPA to predict social unrest via crop yield…

IFTTT, Twitter, FRAUD_la


’The “electronic” society is a special society contained within the wider “geometric” society … the geometric society is a…

IFTTT, Twitter, PeterSjostedtH


Documentary wants: Throbbing Gristle Crass KLF Adrian Sherwood and On U Sound 80s Liverpool Scene Napalm Death The Damned Warp…

IFTTT, Twitter, eops


Damn, my GTD game has completely fallen apart in the last couple of years. My workflow is a global quantum entangled state of…

IFTTT, Twitter, vgr


’Government funding and tourism revenues … have their limitations. But the mayor of Easter Island has come up with an innovative…

IFTTT, Twitter, justinpickard


My touchpoint is that the Air Force measured many dimensions of lots of people and realised there is no ‘average’ human they…

IFTTT, Twitter, debcha


Words of the day: “trophic asynchrony”, “phenological mismatch” - disruptions to established seasonal patterns of migration,…

IFTTT, Twitter, RobGMacfarlane


[cw: media theory] I’m thinking you can make a greimas square of figures for post-broadcast infrastructure: network = horizontal…

IFTTT, Twitter, mckenziewark


People are scrutinizing airline travel as a source of greenhouse gas emissions. Yes, it seems pretty big, and mainly serves the…

IFTTT, Twitter, GlobalEcoGuy


I believe #gopher is making a viral comeback with good reasons; the hacker communities at large are reacting decades after Tim…

IFTTT, Twitter, jaromil


We live in anxious times, teetering on the edge of conflict, vulnerable to algorithmic manipulation. Our new work ’TRIGGER…

IFTTT, Twitter, Superflux


You’re sat on a trolley hurtling down the track. Down one fork is a group of ethicists who think the trolly problem is the only…

IFTTT, Twitter, farbandish


Despite a weekend break from the debate, my attention keeps getting pulled back to the threads arguing that personal action to…

IFTTT, Twitter, bethsawin


For a year I’ve been reporting this story about major climate news, finally breaking today: A new simulation finds that global…

IFTTT, Twitter, nattyover


A fun excerpt from my Voyager research: at one point, NASA realised that starting and stopping the craft’s tape recorder (used…

IFTTT, Twitter, dirigiblebill


Just in US, vehicule idling is burning 40k tons of CO2 daily, thats nearly 15k kt of CO2 yearly and BTC worldwide is 23k kt. So…

IFTTT, Twitter, gigabyted4


Hilarious when people say stuff like “capitalism gave us the internet”. Capitalism has done everything in its power to hold back…

IFTTT, Twitter, existentialcoms


MEDIUM EARTH of @theotolithgroup is generous to the slowness of the eye in gathering cinematic texture. The vastness of the…

IFTTT, Twitter, SonicActs


“Bag of Tricks for Image Classification with Convolutional Neural Networks” is such a great paper. Really clear, and most…

IFTTT, Twitter, jeremyphoward

(via is finally back up t͕̲͉h̺̥a̶̹͈̹t̤̙͉ ̤̝i̗̖͇̳̕ͅͅs҉ ̷̩̖̝̟̤̪ǹ͈̖o̬͕̻ͅț͖̙͞ ḑ͔̮͇e̷̪a̹̙̘͈d̺̖…

IFTTT, Twitter, NyxLandUnlife


A good tool to have, but which i definitely don’t have the skills to make: a bot that wide searches every known public "people…

IFTTT, Twitter, Wolven


Properly stoked to announce that I’ve just signed the contract to write a book with @VersoBooks on systems; if you like how I’ve…

IFTTT, Twitter, gsvoss


A gut feeling for mental health

health, microbiome, depression, KUL, VIB, gut-flora, 2019

The first population-level study on the link between gut bacteria and mental health identifies specific gut bacteria linked to depression and provides evidence that a wide range of gut bacteria can produce neuroactive compounds. Jeroen Raes (VIB-KU Leuven) and his team published these results today in the scientific journal Nature Microbiology. In their manuscript entitled ‘The neuroactive potential of the human gut microbiota in quality of life and depression’ Jeroen Raes and his team studied the relation between gut bacteria and quality of life and depression. The authors combined faecal microbiome data with general practitioner diagnoses of depression from 1,054 individuals enrolled in the Flemish Gut Flora Project. They identified specific groups of microorganisms that positively or negatively correlated with mental health. The authors found that two bacterial genera, Coprococcus and Dialister, were consistently depleted in individuals with depression, regardless of antidepressant treatment. The results were validated in an independent cohort of 1,063 individuals from the Dutch LifeLinesDEEP cohort and in a cohort of clinically depressed patients at the University Hospitals Leuven, Belgium.



python, visualization, data, 2018

Datashader is a graphics pipeline system for creating meaningful representations of large datasets quickly and flexibly. Datashader breaks the creation of images into a series of explicit steps that allow computations to be done on intermediate representations. This approach allows accurate and effective visualizations to be produced automatically without trial-and-error parameter tuning, and also makes it simple for data scientists to focus on particular data and relationships of interest in a principled way.


Worldwide decline of the entomofauna: A review of its drivers

insects, climate-change, biodiversity, 2019, ecology, science, extinction

Biodiversity of insects is threatened worldwide. Here, we present a comprehensive review of 73 historical reports of insect declines from across the globe, and systematically assess the underlying drivers. Our work reveals dramatic rates of decline that may lead to the extinction of 40% of the world’s insect species over the next few decades. […] The main drivers of species declines appear to be in order of importance: i) habitat loss and conversion to intensive agriculture and urbanisation; ii) pollution, mainly that by synthetic pesticides and fertilisers; iii) biological factors, including pathogens and introduced species; and iv) climate change