A research scientist at Google DeepMind working at the intersection of machine learning and computer security.
The RSS's url is : https://nicholas.carlini.com/writing/feed.xml
Please copy to your reader or subscribe it with :
2024-08-01 08:00:00
I don't think that "AI" models [a] (by which I mean: large language models) are over-hyped.
2024-06-24 08:00:00
Yesterday I was forwarded a bunch of messages that Prof. Ben Zhao (a computer science professor [a] A full professor with tenure, so I feel entirely within my rights to call him out here. at the University of Chicago) wrote about me on a public Discord server with 15,000 members, including this gem:
2024-05-06 08:00:00
IEEE SP 2024 (one of the top computer security conferences) has, again, accepted an adversarial example defense paper that is broken with simple attacks. It contains claims that are mathematically impossible, does not follow recommended guidance on evaluating adversarial robustness, and its own figures present all the necessary evidence that the evaluation was conducted incorrectly.
2024-02-19 08:00:00
A benchmark of ~100 tests for language models, collected from actual questions I've asked of language models in the last year.
2024-01-21 08:00:00
How do I pick what research problems I want to solve? I get asked this question often, most recently in December at NeurIPS, and so on my flight back I decided to describe the only piece of my incredibly rudimentary system that's at all a process. I maintain a single file called ideas.txt, where I just append a new entry every time I think of something that might be an interesting topic for a paper. When it's time to pick my next project, I skim through the list, and pick whichever I think is most interesting. or exciting. or important. or whatever I'm looking for at the moment. (Or find something new entirely if nothing looks compelling.)
2023-12-03 08:00:00
So let's suppose you had a 1980s Apple ProFile Hard Drive, and you wanted to recover the data.
2023-09-22 08:00:00
Building a chess bot that queries GPT-3.5-turbo-instruct to play chess at the level of a skilled human player.
2023-04-02 08:00:00
This program is a dependency-free implementation of GPT-2, including byte-pair encoding and transformer inference, in ~3000 bytes of C. I then use this to create something like Chat GPT.
2022-08-17 08:00:00
I recently got back from attending USENIX Security 2022, and someone pointed out to me that it's been five years since I wrote Towards Evaluating the Robustness of Neural Networks (with my at-the-time advisor) and they asked if I had any thoughts on this paper. I didn't respond with that great an answer, but thought it was an interesting question, so I figured I'd write one down here instead. (In fact, I got the same question at IEEE SP earlier this year too, and didn't have a great answer then either.)
2022-06-19 08:00:00
A brief discussion about a tool I use to make rapid iteration in ML research possible.
2022-04-08 08:00:00
A recent paper ('A Roadmap for Big Model') has copied a bunch of text from over a dozen prior papers. This is bad.
2022-02-27 08:00:00
Abstract: Improving digital logic gates on Conway's game of life by allowing 8-bit logic gates instead of boolean logic gates.
2022-01-30 08:00:00
This page contains a few checklists that help prevent embarrassing issues when releasing research papers online (e.g., via arXiv or a conference publication).
2021-12-30 08:00:00
Abstract: An implementation of a minimal CPU on Conway's the Game of Life (an 'unlimited register machine'), and runs at ~10Hz.
2021-03-23 08:00:00
Abstract: This post describes improvemnets made to my prior digital logic gate constructions (e.g., AND/OR/NOT) built on top of Conway's Game of Life, resulting in 100x faster simulations.
2020-12-19 08:00:00
This year I entered in JS13K 2020, a game jam for JavaScript games in under 13KB (total size). I wrote a 3rd-person space shooter game, building on top of game engine I built last year for a doom clone.
2020-12-05 08:00:00
InstaHide (a recent method that claims to give a way to train neural networks while preserving training data privacy) was just awarded the 2nd place Bell Labs Prize (an award for finding solutions to some of the greatest challenges facing the information and telecommunications industry.). This is a grave error.
2020-11-21 08:00:00
For the third year in a row, I participated in JS13k 2021, where you're tasked with making a game in 13kB of JavaScript. Each year I enter participate I try to learn something new I didn't know how to do before. This year's motivation: I wanted to make a multiplayer game with some nontrivial networking aspects. So below you'll find a description of my experience building a MOBA in (barely) under 13kB of JavaScript.
2020-09-15 08:00:00
I recently broke a defense to be published at CCS 2020, and this time I recorded my screen the entire time---all two hours of it. Typically when I break defenses, I'll write a short paper, stick it on arXiv, and then move on. Pedagogically, this isn't very useful. [a] (Don't you worry, I did that again this time, too.) So for this defense I thought I'd try something different.
2020-06-01 08:00:00
Abstract: Using AND/OR/NOT gates built on top of Conway's Game of Life, this post walks through how to construct a actual circuits, for example a 7-segment display.
2020-04-01 08:00:00
Abstract: This post walks through how to construct digital logic gates (AND/OR/NOT) on top of Conway's Game of Life, demonstrating its Turing completeness.
2020-02-20 08:00:00
Abstract: We (again) broke a large collection of published defenses to adversarial examples. Here's how and why.
2019-09-13 08:00:00
This year I entered in JS13K 2019, which asks people to develop games in under 13K of JavaScript. I entered a Doom Clone called ... Yet Another Doom Clone.
2019-08-12 08:00:00
Late last year I decided it would be fun to build a 3D renderer in JavaScript. Recently it got into some sort of finished state and decided to put it here. This isn't so much of a tutorial on how to get there, but rather more of a here's a fun thing to do with nice pictures. But it was interesting to do. So here's that.
2019-06-15 08:00:00
Abstract: A continuously-updating list of all 1000+ papers posted to arXiv about adversarial examples.
2018-07-15 08:00:00
Abstract: This reading list provides an introduction to the field of adversarial examples for machine learning models.
2018-05-26 08:00:00
Abstract: This document contains a collection of advice for performing adversarial example defense evaluations.