Learn with Abdou

Chapter 2

Ask questions of one table

SELECT, WHERE, LIKE, NULL and what top five means

Where you are

Part I, before Exam 1

This chapter goes with Lab 2: SELECT and WHERE.

By the end you can choose columns and rows from one table, match text by its shape, keep a missing value honest, and say exactly what top five means before you ask for it.

Rick & Morty is a table called scripts: 1,905 lines of dialogue from eleven episodes of the show, one row per spoken line. Open the playground, pick Rick & Morty, and run the five-row preview. The first row is id 1: Rick, in the episode Pilot, saying 'Morty! You gotta come on. Jus'... you gotta come with me.' Eleven words, and the table knows that too. wordCount says 11.

That row is the whole chapter in miniature. Before you count anything, rank anything or claim anything, you have to know what one row is. Here one row is one spoken line. It is not one character, because Rick is on 420 rows, and it is not one episode, because Pilot is on 241. Every question this chapter teaches you to ask, which columns, which rows, which text, what to do with a blank, what top five means, starts from that sentence.

Lab 2 asks the same questions of Northshore Coffee, a fictional chain of eight coffee shops along Lake Superior's North Shore, whose owner has a list of questions about her shops, her menu and her staff. You practise on Rick's lines first because the table is big enough to surprise you and small enough to check by hand. The ladders move you to the coffee shops, to a thousand IMDb films and to twelve North American cities once the idea holds.

One row is one spoken line

Open the playground, choose Rick & Morty, and run the five-row preview below. The first row is id 1: Rick, in the episode Pilot, saying 'Morty! You gotta come on. Jus'... you gotta come with me.' Nine columns describe that one line: which season and episode it belongs to, who said it, the words themselves, and how long it is. wordCount says 11. Count the words yourself; the table is right.

SELECT *
FROM scripts
ORDER BY id
LIMIT 5;
Five lines of the Pilot, every column. One row is one spoken line. Open in the playground

That is one row. Before you write any other query, finish this sentence for the table in front of you: one row in scripts represents one spoken line. Not one character: Rick is on 420 rows. Not one episode: Pilot is on 241. A familiar name is not an identifier. The id is, and it is the column you keep while you investigate, because it lets you point at one line and say this one.

SELECT names the columns you want, in the order you want them; FROM names the table. Asking for two columns instead of nine changes the shape of the result, not the number of rows: 1,905 lines stay 1,905 lines. ORDER BY id makes the first four mean the same four every time you run it, and LIMIT 4 keeps the preview short. A preview is a sample. It tells you what the table looks like; it cannot tell you anything about the lines you did not look at.

SELECT id, characterName, episodeName
FROM scripts
ORDER BY id
LIMIT 4;
Four rows, three columns. The same 1,905 lines are still in the table. Open in the playground

So make the claim as narrow as the evidence. Six rows that show only Rick and Morty are six lines and two names. They are not six characters, and they are not the main characters. Say what you saw, and keep the id so a teammate can look at the same six lines.

Rule of thumb. Say what one row is before you count, rank or claim anything.

Going deeper (optional, not on the exam)

The column names in scripts run words together, characterName and wordCount, because the file arrived that way. Northshore Coffee writes first_name and hourly_wage with underscores. Neither style is wrong; the schema panel tells you the exact spelling, and the exact spelling is the one the engine accepts.

SELECT * is for a first look. When you hand a result to someone who asked for three columns, name the three. The result is easier to read, and it does not change shape when a column is added to the table next year.

Name the result

The recap editor does not want a heading called characterName. Write characterName AS speaker and the output column is called speaker. That is all an alias does: it names a column of this result, for this query. Open the schema panel and the table still says characterName. Nothing was renamed.

An expression calculates a column that was never stored. wordCount + 5 AS five_more_words gives 16 for id 1, whose wordCount is 11. Run SELECT wordCount FROM scripts WHERE id = 1 afterwards and you get 11 again. SELECT reads; it never writes. Check an expression against one row you can do in your head, 11 plus 5, before you trust it on 1,905.

SELECT id, characterName AS speaker, wordCount + 5 AS five_more_words
FROM scripts
ORDER BY id
LIMIT 3;
Three rows: 11 becomes 16, 5 becomes 10, 7 becomes 12. The stored word counts stay 11, 5 and 7. Open in the playground

Text joins with ||, not with +. characterName || ' in ' || episodeName AS scene_label gives Rick in Pilot. The single quotes around ' in ' make it a piece of text, spaces included. Leave the quotes off a column name; put them on anything that is literally text.

SELECT characterName || ' in ' || episodeName AS scene_label
FROM scripts
ORDER BY id
LIMIT 5;
One column. The first label is Rick in Pilot, with the spaces the quotes asked for. Open in the playground

Now put quotes where they do not belong: SELECT 'characterName' AS speaker. The engine does not complain. It prints the word characterName on every row, because quoted text is a value, not a column. It ran. That is not the same as being right. When a column repeats one word all the way down, look for quotes that should not be there.

Rule of thumb. An alias names the output and an expression calculates it; neither changes what is stored.

Going deeper (optional, not on the exam)

The browser engine divides two whole numbers as whole numbers: 5 / 2 gives 2, not 2.5. Write 5 / 2.0 when you want the fraction, and runtime / 60.0 when you turn minutes into hours. Chapter 4 comes back to this.

+ adds numbers. Write characterName + episodeName and the engine tries to add two pieces of text and answers 0 rather than complaining. The text operator is ||.

An alias with a space in it needs double quotes: phone AS "Front Desk". Single quotes are for text values. You can sort by an alias, ORDER BY five_more_words, but inside WHERE repeat the expression itself.

Ask for Morty

Ask for Morty's lines and the engine walks the table one row at a time, tests each row, and keeps the row when the test is true. WHERE characterName = 'Morty' keeps 347 of the 1,905. Read a query in the order the engine thinks about it: FROM finds the table, WHERE decides which rows survive, SELECT decides which columns of the survivors you see. Filtering rows and choosing columns are separate decisions, and you may filter on a column you never show.

SELECT id, characterName, episodeName
FROM scripts
WHERE characterName = 'Morty'
ORDER BY id
LIMIT 5;
Morty's first five lines: ids 2, 4, 6, 9 and 11. WHERE chose the rows; SELECT chose the columns. Open in the playground

Morty is text, so it goes in single quotes. Write WHERE characterName = Morty with no quotes and the engine looks for a column called Morty, finds none, and stops: no such column: Morty. Numbers go bare: wordCount >= 20, not '20'. The error is the friendly case. The unfriendly case is a query that runs and answers a different question than the one you meant.

So write the contract before the query. Rick lines with at least 20 words has two tests, and at least says a line with exactly 20 words must pass. Rick has 136 such lines. Write wordCount > 20 instead and you get 125: the eleven lines that sit exactly on the boundary are gone, and nothing on the screen says so. Before you run a filter with a number in it, name one row that should pass and one that should fail, on each side of the line.

SELECT id, characterName, wordCount
FROM scripts
WHERE characterName = 'Rick' AND wordCount = 20
ORDER BY id;
Eleven Rick lines of exactly 20 words. A > keeps none of them; a >= keeps all eleven. Open in the playground

The quick test for a boundary is to ask for the boundary itself. Lines with exactly 20 words exist. If your query is supposed to keep them, you can see whether it does.

Rule of thumb. Write the contract first, then test one row on each side of the boundary.

Going deeper (optional, not on the exam)

<> and != both mean not equal. The course writes <>, and the engine accepts either.

An empty result is not a broken query. This snapshot holds eleven episodes from three seasons, not the whole show, and one table, scripts. If a filter returns nothing, look at the values that exist, with SELECT DISTINCT, before you decide the SQL is wrong. There is no characters table and no season column; the schema panel is the list of what there is.

Rick gets a free pass

AND keeps a row when both tests pass; OR keeps it when at least one does. Now ask for Rick or Morty lines with at least 20 words, and write it the way you said it: characterName = 'Rick' OR characterName = 'Morty' AND wordCount >= 20. It runs. The first row back is id 1, Rick, 11 words. AND binds before OR, so the engine read A OR (B AND C): any Rick line at all, or a Morty line of 20 words or more. Rick got a free pass.

SELECT id, characterName, wordCount
FROM scripts
WHERE characterName = 'Rick' OR characterName = 'Morty' AND wordCount >= 20
ORDER BY id
LIMIT 8;
Eight rows, and the first has 11 words. The word test only applied to Morty. Open in the playground

Parentheses state the rule you meant: (characterName = 'Rick' OR characterName = 'Morty') AND wordCount >= 20. Then test one row from each case before you believe it: Rick with 19 words is out, Rick with 20 is in, Morty with 20 is in, Summer with 30 is out because she is the wrong speaker. A good explanation says why each row is kept or dropped. It does not repeat the keywords.

SELECT id, characterName, wordCount
FROM scripts
WHERE (characterName = 'Rick' OR characterName = 'Morty') AND wordCount >= 20
ORDER BY id
LIMIT 8;
Eight rows, every one with 20 words or more. Ids 21 and 24 sit exactly on the boundary and are kept. Open in the playground

IN is a list of exact values for one column: characterName IN ('Rick', 'Morty') is one test, not two, so it keeps you out of the parentheses trouble. BETWEEN 8 AND 12 includes both ends; it means wordCount >= 8 AND wordCount <= 12 and nothing more. Both are shorthand for comparisons you already know how to write.

A syntax error stops you, which is a kindness. A logic error hands you a neat table. Someone who wants films from 1999 or 2000 writes WHERE year = 1999 OR 2000, and the engine returns every film in the table, because a bare 2000 counts as true and true OR anything is true. Every OR needs a complete comparison on each side, or an IN list. After any repair, look at one row on the boundary and one row that should have been excluded.

Rule of thumb. When AND and OR share a WHERE, put parentheses around the OR.

Going deeper (optional, not on the exam)

NOT flips a test: NOT city = 'Duluth' is the same as city <> 'Duluth'. NOT BETWEEN 100 AND 200 keeps the values outside the range and nothing else: a row whose value is unknown fails both the range and its opposite. Missing is not zero, two sections on, explains why.

Read a mixed condition as a small decision tree: which test is asked first, and what each branch leads to. Draw it on paper for a teammate. If you cannot, the parentheses are in the wrong place.

Where is Rick in the title?

= asks for an exact stored value. LIKE asks for a shape. Rick is in the title of four episodes in this snapshot: A Rickle in Time, Pickle Rick, Rick Potion 9 and The Rickshank Rickdemption. Ask for episodeName LIKE 'Rick%' and you get one of them, Rick Potion 9, because 'Rick%' means begins with Rick. '%Rick' means ends with Rick, and '%Rick%' means Rick anywhere, which is what the question wanted.

SELECT DISTINCT episodeName
FROM scripts
WHERE episodeName LIKE '%Rick%'
ORDER BY episodeName;
Four titles. 'Rick%' would have found one of them. Open in the playground

% stands for any run of characters, including none, so Rick on its own also matches 'Rick%'. _ stands for exactly one character. characterName LIKE 'R___' is four characters that start with R: one speaker label, Rick. Change it to 'R%' and Rick J-22 and Riq IV come along. Pick the wildcard that says what you mean about length.

SELECT DISTINCT characterName
FROM scripts
WHERE characterName LIKE 'R___'
ORDER BY characterName;
One row. Four characters, the first an R. Open in the playground

A pattern matches characters, not meaning. A Rickle in Time matched because of four letters in a row, and a title with Rick in it says nothing about who speaks the lines. The four Rick-titled episodes hold 714 lines; 113 of them are Rick's. If a teammate claims every line in those episodes is Rick's, you do not need a second matching row. You need one line that is not his, and there are 601 to choose from. Keep claims about titles, speakers and dialogue separate, and say which one your evidence is about.

Rule of thumb. LIKE matches characters, not meaning.

Going deeper (optional, not on the exam)

The browser engine's LIKE ignores the case of plain letters: LIKE 'rick' finds Rick's 420 lines, while = 'rick' finds none. Other databases make their own choices here, so do not carry the habit across without checking.

To put an apostrophe inside text, double it: 'Schindler''s List'. To search for a literal underscore or percent sign, name an escape character: LIKE '%!_%' ESCAPE '!'. Both are reference details, not exam material.

characterName = 'Rick%' is not a pattern. With = the percent sign is an ordinary character, and no name ends in one.

One of each

List characterName for every line and you get 1,905 rows, most of them Rick and Morty over and over. Put DISTINCT in front and you get 48: one row per different name. Nothing was deleted. DISTINCT removes repeats from this result, and it changes what one row means: a row is now a speaker label, not a line.

SELECT DISTINCT characterName
FROM scripts
ORDER BY characterName;
48 rows, from Agency Director to Young Rick. One row is one speaker label. Open in the playground

DISTINCT looks at the whole list of columns you selected. DISTINCT characterName, episodeName gives 104 rows, one per speaker-and-episode pair: Rick in Pilot once, Rick in Lawnmower Dog once. Add id to the list and every row is distinct again, all 1,905, because no two lines share an id. After any DISTINCT, say out loud what one row now represents.

DISTINCT is a reporting choice, not a repair. If a result repeats and you did not expect it, ask which columns should define a duplicate before you reach for the keyword. And sort a distinct list by the column it lists, so two people comparing their speaker directories see the same order.

Rule of thumb. DISTINCT changes what one row represents; say what it is now.

Missing is not zero

Rick's table has a number in every wordCount. Northshore Coffee's menu does not: the Chocolate Croissant has no calorie count on file, and neither do five other items. That blank is NULL, and NULL means unknown. It is not 0, it is not an empty piece of text, and it is not the word NULL. A croissant with an unknown calorie count is not a croissant with no calories.

Ask for calories = NULL and the engine returns no rows and no error. The comparison does not come out true or false; it comes out unknown, and WHERE keeps only rows whose test is true. The test that works is IS NULL, and its opposite is IS NOT NULL. The same goes for <> NULL, which keeps nothing either.

SELECT name, calories
FROM menu_items
WHERE calories IS NULL
ORDER BY name;
Six items with no calorie count. WHERE calories = NULL finds none of them. Open in the playground

An unknown value fails every ordinary test, in both directions. calories < 100 keeps 10 menu items, calories >= 100 keeps 29, and six items are in neither list: 10 plus 29 plus 6 is the whole menu of 45. BETWEEN and NOT BETWEEN split the same way. If a rule is supposed to include the unknowns, say so in the query: calories < 100 OR calories IS NULL.

Report the gap instead of filling it. A film with no gross on file did not earn nothing; 169 of the thousand IMDb films have no gross at all, and a report that quietly leaves them out, or quietly counts them as zero, is wrong twice. Say how many rows are unknown, next to the number you computed from the rest.

Rule of thumb. Unknown is not zero: test for it with IS NULL, and say how many rows are unknown.

Going deeper (optional, not on the exam)

NOT turns unknown into unknown, so NOT (calories < 100) keeps the same 29 items that calories >= 100 keeps and still skips the six. NOT IN with a NULL inside the list is worse: it can keep nothing at all. Chapter 8 returns to that when it builds sets of ids.

COALESCE can print a fallback in place of a blank. Replacing an unknown calorie count with 0 makes a claim about the croissant that nobody checked. Chapter 4 shows where a fallback is honest and where it is not.

Define what first and top mean

ORDER BY wordCount DESC puts the longest line first; ASC, the default, puts the shortest first. LIMIT 5 then keeps the first five of that order. Without ORDER BY, LIMIT 5 keeps the first five rows the engine happened to reach. In this file that is usually id order, and nothing promises it. Five rows with no ORDER BY is a sample. It is not a top five.

SELECT id, title, imdb_score
FROM movies
ORDER BY imdb_score DESC, id
LIMIT 5;
The five highest scores with a tie-break. Ids 3, 4 and 5 all score 9.0; the id decides their order. Open in the playground

Ties need a rule. Three IMDb films share a score of 9.0, and ORDER BY imdb_score DESC alone lists those three in whatever order the engine met them. ORDER BY imdb_score DESC, id breaks the tie by id, so the same five films come back in the same order every time. The tie-break is a second column after a comma; make it one that is unique, and the result is repeatable.

Exactly five and everyone tied for fifth are different requests. LIMIT 5 gives you exactly five, and a tie-break decides which of the tied rows make the cut. It never pulls the rest of the tie in. When someone asks for a top five, ask back which of the two they mean.

Describe the population before you rank it. Rick's five longest lines filters to Rick first, then sorts: the longest is id 241, at 196 words. That is one line. It does not tell you who talks most across the show; that is a question about totals per speaker, and it waits for the next chapter.

SELECT id, characterName, wordCount
FROM scripts
WHERE characterName = 'Rick'
ORDER BY wordCount DESC, id
LIMIT 5;
Rick's five longest lines, longest first. Three lines tie at 71 words; the tie-break keeps id 109 and leaves ids 443 and 778 for sixth and seventh place. Open in the playground

OFFSET skips rows of the ordered result: LIMIT 5 OFFSET 5 is lines six to ten, the second page. The playground shows at most 200 rows and labels a result that was cut off. A display that stops at 200 is not a count of 200; narrow the question, or wait for COUNT in chapter 3.

Rule of thumb. No ORDER BY, no top. Add a tie-break, then LIMIT.

Going deeper (optional, not on the exam)

LIMIT without ORDER BY is not a random sample either. The engine returns rows in the order it reads them, which depends on how the table is stored and on the plan it picks, and that changes when the table changes. Treat it as some five rows.

You can sort by an alias: ORDER BY five_more_words DESC works. ORDER BY always runs before LIMIT and OFFSET, whatever order you write them in, and DISTINCT does not sort anything.

Concepts in this chapter

6 concepts

Each concept has its reading and a ladder of six rungs, from watching a question answered to transferring the idea to new data. The rungs below show your saved practice: reached, in progress or not yet.

  1. Concept 1 · SELECT, FROM, ORDER BY id and LIMIT as a preview

    Meet the table

    • Finish the sentence one row in scripts represents, and keep the id that proves it
    • Choose the columns a question needs and predict the rows and columns that come back
    • Say what a five-row preview shows and what it cannot show
    1. 1Watch it answered: not yet
    2. 2Fill the blank: not yet
    3. 3Predict the result: not yet
    4. 4Write it yourself: not yet
    5. 5Repair it: not yet
    6. 6Transfer it: not yet

     

  2. Concept 2 · aliases, expressions and text labels

    Name the result

    • Give an output column a heading with AS and say what did not change
    • Calculate a new column from a stored one and check it against one row
    • Tell a column name from a quoted piece of text, and spot the query that confuses them
    1. 1Watch it answered: not yet
    2. 2Fill the blank: not yet
    3. 3Predict the result: not yet
    4. 4Write it yourself: not yet
    5. 5Repair it: not yet
    6. 6Transfer it: not yet

     

  3. Concept 3 · WHERE, boundaries, AND, OR, parentheses, IN and BETWEEN

    Ask for Morty

    • Write the contract before the query and name one row on each side of a boundary
    • Combine AND and OR with the parentheses that say what you meant, and use IN and BETWEEN as shorthand
    • Tell a syntax error from a logic error, and find the row that proves a running query wrong
    1. 1Watch it answered: not yet
    2. 2Fill the blank: not yet
    3. 3Predict the result: not yet
    4. 4Write it yourself: not yet
    5. 5Repair it: not yet
    6. 6Transfer it: not yet

     

  4. Concept 4 · LIKE, %, _ and DISTINCT

    Where is Rick in the title?

    • Choose = or LIKE on purpose, and pick % or _ for what you mean about length
    • Say which claim a text match is evidence for, and find the one row that refutes a bigger claim
    • Say what one row represents after DISTINCT, with one column or two
    1. 1Watch it answered: not yet
    2. 2Fill the blank: not yet
    3. 3Predict the result: not yet
    4. 4Write it yourself: not yet
    5. 5Repair it: not yet
    6. 6Transfer it: not yet

     

  5. Concept 5 · IS NULL and IS NOT NULL in a filter

    Missing is not zero

    • Find the rows with an unknown value using IS NULL, and say why = NULL finds none
    • Predict which rows an ordinary comparison drops when a value is unknown
    • Report how many rows are unknown instead of treating them as zero
    1. 1Watch it answered: not yet
    2. 2Fill the blank: not yet
    3. 3Predict the result: not yet
    4. 4Write it yourself: not yet
    5. 5Repair it: not yet
    6. 6Transfer it: not yet

     

  6. Concept 6 · ORDER BY, DESC, ties, LIMIT and OFFSET

    Define what first and top mean

    • Sort a result on purpose, largest or smallest first, and add a tie-break that makes it repeatable
    • Say why LIMIT without ORDER BY is a sample and not a top five
    • Filter the population before ranking it, and page through an ordered result with OFFSET
    1. 1Watch it answered: not yet
    2. 2Fill the blank: not yet
    3. 3Predict the result: not yet
    4. 4Write it yourself: not yet
    5. 5Repair it: not yet
    6. 6Transfer it: not yet

     

Chapter review

Key terms

SELECT
The clause that names the columns of the result, in order.
FROM
The clause that names the table the rows come from.
WHERE
The clause that tests each row and keeps it when the test is true.
condition
One test on a row, such as wordCount >= 20; a WHERE is one condition or several joined by AND and OR.
boundary
The value where > and >= disagree; at least includes it, more than does not.
alias
A name given with AS to an output column of this query; the stored column keeps its name.
expression
A value calculated for each row, such as wordCount + 5, that was never stored.
LIKE
A test that matches text by its shape, with % for any run of characters and _ for exactly one.
DISTINCT
The keyword that drops repeated rows from a result and changes what one row represents.
NULL
An unknown value: not zero, not empty text, and never equal to anything, so it needs IS NULL.
ORDER BY
The clause that sorts the result, ASC by default, DESC for largest first.
tie-break
A second sort column, usually the id, that decides the order of rows with equal values.
LIMIT and OFFSET
LIMIT keeps the first N rows of the ordered result; OFFSET skips rows before it starts.

Check your understanding

Short questions you can answer from the chapter. A choice shows its feedback at once; a sentence shows a model answer after you save it.

Opening your saved work…

Sign in to save across devices → · Download your guest work before switching accounts.

Question 1Which row passes characterName = 'Rick' OR characterName = 'Morty' AND wordCount >= 20 but fails the same condition with parentheses around the OR?
Question 2Six menu items have no calorie count. How many rows does SELECT name FROM menu_items WHERE calories = NULL return?
Question 3SELECT id, characterName, wordCount FROM scripts LIMIT 5 returns five rows. What are they?

Applied to Lab 2

Warm up on Northshore Coffee with the first line of the lab companion: name and city for all eight shops, ordered by id. Then take one of the owner's numbered questions and write your explanation twice. Weak: the query ran and returned rows. Strong: the condition in words, one row that passed and why, one row on the boundary that did not, and the row count. Hand in the strong one.

Sources and further reading

  • Week 2 notes: The SELECT Statement (aliases, calculations, string concatenation, DISTINCT, LIMIT and OFFSET) and Filtering with WHERE (comparison operators, BETWEEN, IN, IS NULL, LIKE, AND, OR and NOT)
  • Pre-Lab 2, parts 1 and 3
  • The first interactive lecture and the Multiverse Briefing Room team workshop on Rick & Morty
  • The foundation seminar readings and the SQL foundations practice page

All chapters