MySQL "Incorrect string value" / emoji needs utf8mb4

MySQL's historic charset named "utf8" only stores three bytes per character. Emoji need four, so inserts die with Error 1366 — or silently become question marks.

utf8 vs utf8mb4 in MySQL

CharsetMax bytes / characterStores emoji?Notes
utf8 (alias utf8mb3)3NoHistoric MySQL "utf8". Rejects 4-byte sequences.
utf8mb44YesReal UTF-8. Use this everywhere.
latin11NoWestern European only; causes mojibake with UTF-8 clients.

Emoji live on Unicode planes above U+FFFF. In UTF-8 they are 4 bytes. A utf8/utf8mb3 column cannot hold them, so MySQL raises:

ERROR 1366 (HY000): Incorrect string value: '\xF0\x9F\x98\x80' for column 'body' at row 1

With non-strict SQL modes some setups store ? instead — permanent data loss. See the crime-scene version in Wirebook's emoji lesson.

Convert existing data

-- Database default for new tables
ALTER DATABASE yourdb CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

-- One table (rewrites every text column)
ALTER TABLE your_table CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

-- Or a single column
ALTER TABLE your_table MODIFY body TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

Connection charset (the silent trap)

Even with utf8mb4 columns, a connection still on utf8 will fail on the way in. Set it every time:

SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci;
-- or in my.cnf / mysqld.cnf
[client]
default-character-set = utf8mb4
[mysql]
default-character-set = utf8mb4
[mysqld]
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci

ORM / framework notes

StackWhat to set
DjangoOPTIONS: {"charset": "utf8mb4"} in DATABASES; older docs mentioning utf8 are wrong for emoji.
Railsencoding: utf8mb4 and collation: utf8mb4_unicode_ci in database.yml; use the utf8mb4 migration helpers.
PrismaURL query ?charset=utf8mb4 (or connection init). Schema @db.VarChar length may need revisiting for unique indexes.
Node (mysql2 / Sequelize)charset: 'utf8mb4' in the pool config. Do not use the bare utf8 alias.
Python (PyMySQL / SQLAlchemy)charset=utf8mb4 in the URL or connect args. SQLAlchemy: ?charset=utf8mb4.
PHP PDOPDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8mb4", or DSN charset.

Quick checklist

  1. Reproduce with a single emoji insert; confirm Error 1366 or ?.
  2. SHOW FULL COLUMNS FROM your_table; — look for utf8 / utf8mb3.
  3. SHOW VARIABLES LIKE 'character_set%'; — client/connection/database should all say utf8mb4.
  4. ALTER + fix the driver/ORM.
  5. Re-test. For mystery blobs, What is this?; for UTF-8 byte length, Base64. JSON APIs: validate with the JSON formatter (JSON is UTF-8).

Related

Emoji shows as □ / tofu · Mojibake / ðŸ / é · Invisible zero-width characters · Base64 is not encryption · Wirebook emoji lesson · Anvil text X-ray API · UnicodeDecodeError utf-8

Questions

What does MySQL Incorrect string value mean for emoji?

Error 1366 usually means the column or connection is using the old 3-byte utf8 charset. Emoji need utf8mb4.

What is the difference between utf8 and utf8mb4?

In MySQL, utf8 (utf8mb3) stores at most 3 bytes per character and rejects emoji. utf8mb4 is real UTF-8 with a 4-byte maximum.

How do I convert a MySQL table to utf8mb4?

ALTER TABLE t CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; also set the connection and ORM charset to utf8mb4.

Why do unique indexes break after switching to utf8mb4?

InnoDB's old 767-byte index limit: VARCHAR(255) utf8mb4 can be 1020 bytes. Shorten the column or use a prefix index (often VARCHAR(191)).

Can I recover emoji already stored as ?

No. The 4-byte sequence was discarded on write. Only mojibake (é / ðŸ) is usually recoverable; see the mojibake guide.

Published .

Try it

Related guides

Newest guides