The fourth book in ABC Books is McGuffey's Third Eclectic Reader, Revised Edition (1879): 77 of its 79 lessons. It's the purple book on the shelf at https://games.503mail.com/abcbooks/. Every lesson plays like the others: new words, listen, read aloud, trace in print and cursive, and the blackboard. This post covers what's special about this book and how to add it. The app itself is rebuilt in the Primer post. Every block has a Copy button.
The book
- McGuffey's Third Eclectic Reader, Revised Edition, copyright 1879, from Project Gutenberg #14766 (public domain).
- Longer stories, poems and short plays: "The Shepherd Boy", "Johnny's First Snowstorm", "Let It Rain" (a play between Rose and her father), "The Truant", the beaver, "The Wind and the Sun", "Which Loved Best?", "The Bundle of Sticks", the mountain sister Jeannette, the Himalayas, and the closing poem "Good Night". Many poems end with their author (John G. Whittier, Mrs. Follen and others); the app keeps it as the lesson's author.
- Two lessons are skipped: Lesson 6 ("Lend a Hand") and Lesson 23 ("Sunset") have only their title in Gutenberg's copy. In the printed book they were most likely script (handwriting) pictures. The other lessons keep the book's own numbers, so Lesson 7 is still Lesson 7.
- About 8,300 different things to say per speed, recorded in ETI-Eloquence: about 190 MB for both speeds together.
New words are worked out automatically
The printed Third Reader lists the new words above each lesson, and defines them in the second half of the book. Gutenberg's copy leaves all of that out. So make-book.ps1 works them out: a word is "new" the first time it shows up, counting everything in the Primer, the First and Second Readers and the earlier lessons of this book. Each lesson shows the first 10 new words in reading order, for example Lesson 9: beaver, chiefly, America, including, flat, paddle, shaped, foot, chestnut, skin. Names keep their capital letter (Jeannette, Isaac Newton); other words are shown in lowercase even when the sentence starts with them.
What the converter had to clean up
On top of the earlier posts (First Reader, Second Reader), this book needs a "prose" mode in make-book.ps1:
- Printed page headers like "18 ECLECTIC SERIES." and "THIRD READER. 19" sit in the middle of sentences, together with page numbers like "(13)" and stray marks like "3,". They're removed, and the sentence is joined up again across the page break.
- Numbered paragraphs run on without blank lines, so every "1.", "2."... starts a new paragraph.
- Plays: a name that starts two or more lines ("Rose. See how it rains!") is a speaker, and becomes "Rose: See how it rains!". "Mr." and "Mrs." are never taken for speakers.
- The author's name under a poem ("Mrs. Follen.") is kept as the author, not read as a line.
- Old spellings: "do n't" becomes "don't" and "'T is" becomes "'Tis", so Eloquence reads them right.
- Titles ending in "?" ("WHO MADE THE STARS?") are titles too.
- One heading has a double dot ("LESSON XXX..").
- Pictures: this book has no picture notes, so the emoji is guessed from the title and the first lines (a cow for "The Shepherd Boy", a snowman for the snowstorm), matching whole words only, so "been" doesn't become a bee. Otherwise it's a book 📖.
- The first three books still come out exactly the same; I check that after every change to the converter.
Add this book to ABC Books
You need the app from the Primer post and its C:\ABCBooks folder on the PC (with ETI-Eloquence installed).
1. The converter
If your C:\ABCBooks\tools\make-book.ps1 is older than this post, replace it. Proxmox host shell, then copy it to the PC the same way as in the Primer post (the scp and arrange-rebuild.ps1 steps):
mkdir -p /root/abcbooks-rebuildcat > /root/abcbooks-rebuild/make-book.ps1 <<'EOF'
# Builds public/books/<book>.json from a Project Gutenberg text of a McGuffey Reader.
# Usage: powershell -ExecutionPolicy Bypass -File ABCBooks\tools\make-book.ps1 -Book primer1881
# powershell -ExecutionPolicy Bypass -File ABCBooks\tools\make-book.ps1 -Book first1879
param([ValidateSet('primer1881', 'first1879', 'second1879', 'third1879', 'fourth1879', 'fifth1879', 'sixth1879', 'spell1867')] [string]$Book = 'primer1881')
$root = Split-Path -Parent (Split-Path -Parent $MyInvocation.MyCommand.Path)
$books = @{
# Spell & Read: the 1867 Pictorial Primer, hand-corrected from a scan into source/spell1867.txt (no Gutenberg text exists)
spell1867 = @{ hand = $true; file = 'spell1867.txt'; title = 'Spell & Read'; sub = 'Pictorial Primer, 1867'; color = '#5b6f3a'
source = 'McGuffey''s Pictorial Eclectic Primer, 1867 (archive.org mcguffeyspictori00mcgu, public domain), hand-corrected' }
primer1881 = @{ gutenberg = 14642; file = 'primer.txt'; title = "McGuffey's Eclectic Primer"; sub = 'Revised Edition, 1881'; color = '#b5562c'
end = 'SLATE EXERCISES'; pics = @{ 27 = '🏞️'; 46 = '✏️' } }
first1879 = @{ gutenberg = 14640; file = 'first.txt'; title = "McGuffey's First Eclectic Reader"; sub = 'Revised Edition, 1879'; color = '#2c6fb5'
end = 'PHONIC CHART.'; pics = @{} }
second1879 = @{ gutenberg = 14668; file = 'second.txt'; title = "McGuffey's Second Eclectic Reader"; sub = 'Revised Edition, 1879'; color = '#3f8f4f'
end = '*** END OF THE PROJECT GUTENBERG'; pics = @{} }
# prose = later Readers: no word lists in Gutenberg's copy (new words are worked out), printed page headers to remove,
# numbered paragraphs that run on without blank lines, plays ("Rose. ...") and an author line under poems
third1879 = @{ gutenberg = 14766; file = 'third.txt'; title = "McGuffey's Third Eclectic Reader"; sub = 'Revised Edition, 1879'; color = '#7b4fb5'
end = '*** END OF THE PROJECT GUTENBERG'; pics = @{}; prose = $true; pageRx = 'ECLECTIC SERIES|THIRD READER'
after = @('primer1881', 'first1879', 'second1879') }
# Fourth Reader: headings carry the title ("I. PERSEVERANCE. (25)"), and each lesson ends with DEFINITIONS (the book's
# own new words, with meanings), EXERCISES (questions to talk about) and sometimes NOTES
fourth1879 = @{ gutenberg = 14880; file = 'fourth.txt'; title = "McGuffey's Fourth Eclectic Reader"; sub = 'Revised Edition, 1879'; color = '#b5892c'
end = '*** END OF THE PROJECT GUTENBERG'; pics = @{}; prose = $true; pageRx = 'ECLECTIC SERIES|FOURTH READER'
start = 'I. PERSEVERANCE. (25)'; tocStart = 'SELECTIONS IN PROSE AND POETRY.'; tocEnd = 'LIST OF ILLUSTRATIONS'
headRx = '(?m)^([IVXLCl]+)[.:][ \t]+([A-Z][^a-z\n]*?(?:\((?:Concluded|Continued)\.?\))?[^a-z\n]*?)[ \t]*(?:\(\d+\))?[ \t]*$'
after = @('primer1881', 'first1879', 'second1879', 'third1879') }
# Fifth Reader: like the Fourth, but headings have no page number ("II. THE BLUEBELL.") and the first one is scanned as "1."
fifth1879 = @{ gutenberg = 15040; file = 'fifth.txt'; title = "McGuffey's Fifth Eclectic Reader"; sub = 'Revised Edition, 1879'; color = '#2c9bb5'
end = '*** END OF THE PROJECT GUTENBERG'; pics = @{}; prose = $true; pageRx = 'ECLECTIC SERIES|FIFTH READER'
start = '1. THE GOOD READER.'
headRx = '(?m)^(\d+|[IVXLCl]+)[.:][ \t]+(["\u201c]?[A-Z][^a-z\n]*?(?:\((?:Concluded|Continued)\.?\))?[^a-z\n]*?)[ \t]*(?:\(\d+\))?[ \t]*$'
tocEnd = 'LIST OF ILLUSTRATIONS'
after = @('primer1881', 'first1879', 'second1879', 'third1879', 'fourth1879') }
# Sixth Reader: headings with page numbers like the Fourth; the lessons start after the chapters on reading aloud
sixth1879 = @{ gutenberg = 16751; file = 'sixth.txt'; title = "McGuffey's Sixth Eclectic Reader"; sub = 'Revised Edition, 1879'; color = '#8f3f5f'
end = '*** END OF THE PROJECT GUTENBERG'; pics = @{}; prose = $true; pageRx = 'ECLECTIC SERIES|SIXTH READER'
start = 'I. ANECDOTE OF THE DUKE OF NEWCASTLE.'
headRx = '(?m)^(\d+|[IVXLCl]+)[.:][ \t]+(["\u201c]?[A-Z][^a-z\n]*?(?:\((?:Concluded|Continued)\.?\))?[^a-z\n]*?)[ \t]*(?:\(\d+\))?[ \t]*$'
tocEnd = 'LIST OF ILLUSTRATIONS'
after = @('primer1881', 'first1879', 'second1879', 'third1879', 'fourth1879', 'fifth1879') }
}
$cfg = $books[$Book]
$pics = [ordered]@{ 'rat'='🐀'; 'cat'='🐈'; 'kitten'='🐱'; 'dog'='🐕'; 'frog'='🐸'; 'chick'='🐤'; 'hen'='🐔'; 'fox'='🦊'; 'duck'='🦆'; 'owl'='🦉'; 'bird'='🐦'; 'cow'='🐄';
'rabbit'='🐇'; 'pig'='🐖'; 'lamb'='🐑'; 'kite'='🪁'; 'sled'='🛷'; 'top'='🎩'; 'ball'='⚽'; 'squirrel'='🐿️'; 'bee'='🐝'; 'boat'='⛵';
'ship'='⛵'; 'skat'='⛸️'; 'horse'='🐎'; 'train'='🚂'; 'goat'='🐐'; 'burning'='🔥'; 'school'='🏫'; 'quail'='🐦'; 'baby'='👶'; 'lighthouse'='🗼';
'sheep'='🐑'; 'snow'='☃️'; 'fishing'='🎣'; 'doll'='🪆'; 'sawing'='🪚'; 'lamp'='🪔'; 'seashore'='🏖️'; 'blind'='🦯'; 'mill'='🏞️';
'sunset'='🌅'; 'woman'='👩'; 'wagon'='🐎'; 'acorn'='🌰'; 'drum'='🥁'; 'lake'='🌊'; 'writing'='✏️'; 'man'='👴'; 'girl'='👧'; 'boy'='👦'; 'children'='👧' }
if ($cfg.hand) {
# Hand-corrected book (source/<file>): "= n" starts a lesson, "words:" is its spelling list, "?" lines are notes, "#" comments
$lessons = @(); $o = $null
foreach ($ln in [IO.File]::ReadAllLines("$root\source\$($cfg.file)")) {
$l = $ln.Trim()
if (-not $l -or $l.StartsWith('#')) { continue }
if ($l -match '^=\s*(\d+)') { if ($o) { $lessons += $o }; $o = [ordered]@{ n = [int]$Matches[1]; pic = '📖'; picText = ''; words = @(); letters = @(); lines = @() }; continue }
if ($l -match '^words:\s*(.*)$') { $o.words = @($Matches[1] -split '\s+' | Where-Object { $_ }); continue }
if ($l.StartsWith('?')) { $o.note = $l.TrimStart('?', ' '); continue }
$o.lines += $l
}
if ($o) { $lessons += $o }
foreach ($o in $lessons) {
if (-not $o.words.Count) { # no list printed: spell the lesson's own words
$w = @(); foreach ($m in [regex]::Matches(($o.lines -join ' '), "[A-Za-z]+(?:'[A-Za-z]+)?")) { $x = if ($m.Value -ceq 'I') { 'I' } else { $m.Value.ToLower() }; if ($w -notcontains $x) { $w += $x } }
$o.words = @($w | Select-Object -First 12)
}
$txt0 = $o.lines -join ' '
foreach ($key in $pics.Keys) { if ($txt0 -match "(?i)\b$key(?:s|es)?\b") { $o.pic = $pics[$key]; break } }
}
$out = [ordered]@{ title = $cfg.title; sub = $cfg.sub; color = $cfg.color; spell = $true; source = $cfg.source; lessons = $lessons }
[IO.File]::WriteAllText("$root\public\books\$Book.json", ($out | ConvertTo-Json -Depth 6), (New-Object Text.UTF8Encoding $false))
"$($lessons.Count) lessons"
$lessons | ForEach-Object { "{0,2} {1} w[{2}] lines={3} {4}" -f $_.n, $_.pic, ($_.words -join ' '), $_.lines.Count, $(if ($_.note) { '? ' + $_.note } else { '' }) }
return
}
if (-not (Test-Path "$root\source\$($cfg.file)")) {
New-Item -ItemType Directory -Force "$root\source" | Out-Null
Invoke-WebRequest "https://www.gutenberg.org/cache/epub/$($cfg.gutenberg)/pg$($cfg.gutenberg).txt" -OutFile "$root\source\$($cfg.file)" -UseBasicParsing
}
$txt = [IO.File]::ReadAllText("$root\source\$($cfg.file)") -replace "`r", ''
$headRx = '(?m)^(?:Lesson 1|LESSON [IVXLC]+\.{0,2}(?:\s*-*\s*REVIEW\.?)?)\s*$' # Third Reader has one "LESSON XXX.."
if ($cfg.headRx) { $headRx = $cfg.headRx }
$body = if ($cfg.start) { $txt.Substring($txt.LastIndexOf($cfg.start)) } else { $txt.Substring([regex]::Match($txt, $headRx).Index) }
$body = $body.Substring(0, $body.IndexOf($cfg.end))
$body = $body -replace '(?m)^SLATE WORK\.?\s*$', ''
if ($cfg.prose) {
# printed page headers ("18 ECLECTIC SERIES.") and page numbers, with the blank lines around them, so sentences join across pages
$body = [regex]::Replace($body, "\n[ \t]*\n?[ \t]*\d*[ \t]*(?:$($cfg.pageRx))\.?[ \t]*\d*[ \t]*\n(?:[ \t]*\n)*", "`n")
$body = $body -replace '(?m)^[ \t]*\(\d+\)[ \t]*\n', '' -replace '(?m)^[ \t]*\d+[,.]?(?:[ \t]*\d+[,.]?)*[ \t]*\n', ''
$body = $body -creplace "(?m)^[ \t]*\d*[ \t]*(?:$($cfg.pageRx))[.,]?[ \t]*\d*[ \t]*\n", '' # leftover headers ("62 ECLECTIC SERIES,")
$body = $body -creplace "(\w) n't\b", '$1n''t' -creplace "'T is\b", "'Tis" # old spelling: "do n't" -> "don't"
}
$heads = [regex]::Matches($body, $headRx)
# the text of each lesson runs from the end of its heading to the next heading (Regex.Split would also return captured groups)
$parts = @($body.Substring(0, $heads[0].Index))
for ($i = 0; $i -lt $heads.Count; $i++) { $s = $heads[$i].Index + $heads[$i].Length; $e = if ($i + 1 -lt $heads.Count) { $heads[$i + 1].Index } else { $body.Length }; $parts += $body.Substring($s, $e - $s) }
$sounds = 'th','sh','ch','ng','ew','oy','ow','wh','ou','oo','sk','qu','oi','gh','ck'
$romanVal = @{ I=1; V=5; X=10; L=50; C=100 }
# scanned "LVll." = LVII, so a lowercase l counts as I
function Roman($r) { if ($r -match '^\d+$') { return [int]$r }; $r = $r -creplace 'l', 'I'; $t = 0; for ($i = 0; $i -lt $r.Length; $i++) { $v = $romanVal["$($r[$i])"]; if ($i + 1 -lt $r.Length -and $romanVal["$($r[$i+1])"] -gt $v) { $t -= $v } else { $t += $v } }; $t }
function Sentences($para) {
# split a paragraph into kid-sized sentences; keep quotes and "--" asides with their sentence
$p = (($para -replace '\s+', ' ').Trim()) -creplace '^(?:\d+|I)\.\s+(?=["A-Z])', ''
$out = [regex]::Split($p, '(?<!\b(?:Mr|Mrs|Dr|St)\.)(?<=[.!?]"?)\s+(?=["A-Z])') | Where-Object { $_ }
# re-join pieces that split inside a quotation
$merged = @(); $buf = ''
foreach ($s in $out) { $buf = if ($buf) { "$buf $s" } else { $s }; if ((($buf -split '"').Count - 1) % 2 -eq 0) { $merged += $buf; $buf = '' } }
if ($buf) { $merged += $buf }
$merged | ForEach-Object { $s = $_.Trim(); if ($s -match '[A-Za-z]$') { $s += '.' }; $s }
}
# vocabulary words carry accent marks and syllable breaks in the later lessons: cun'ning, be cause', did n't
function IsPoem($para) {
$ls = @($para.Trim() -split '\n' | Where-Object { $_.Trim() })
# poem lines are short and each starts with a capital; wrapped prose goes on in lowercase
if ($para -match '\n\s+\S') { return $true }
if ($ls.Count -lt 2 -or ($ls | Measure-Object -Property Length -Maximum).Maximum -gt 55) { return $false }
$low = @($ls | Where-Object { $_.Trim() -cnotmatch '^["''\u2018\u201c(]{0,2}[A-Z]' }).Count
($low -eq 0) -or ($ls.Count -ge 4 -and $low -le 1)
}
function CleanWord($t) {
$t = $t -replace '\s+', ''
if ($t -match "(n't|'s|'ll|'ve|'re|'d|'m)$" -or $t -match "^'tis$") { return ($t -replace "'(?=.*')", '') } # keep a real contraction's apostrophe
$keep = $t.TrimEnd("'")
if ($keep -match "^(o'clock|o'er|ne'er|e'er|e'en)$") { return $keep } # old poetic words keep their apostrophe
$t -replace "'", ''
}
function VocabTokens($ln) {
$ln = $ln -replace '\s*\([^)]*\)', '' # pronunciation notes: (wuns), (a gen'), (as f)
$cols = if ($ln -match '\S\s{2,}\S') { $ln.Trim() -split '\s{2,}' } else { $ln.Trim() -split '\s+' }
foreach ($c in $cols) {
if (-not $c) { continue }
if ($c -match '\s') {
if ($c -match "'") { CleanWord $c } # syllables of one word: be cause'
elseif ($c -match '^(a|an|the)\s') { } # phrase like "a black hen": not a new word
else { $c -split '\s+' | ForEach-Object { CleanWord $_ } } # two words in one column: night owl
} else { CleanWord $c }
}
}
$nums = @()
if ($cfg.headRx) {
# A real lesson heading has a page number ("(74)") or is the next number in order. That skips a lesson's own
# numbered parts ("II. SUMMER.") and survives misprinted numerals ("XVV.", a second "LXX."): lessons are numbered in order.
$keep = @(); $prev = 0
# up to 2 lessons may be missing between two headings; a heading with a page number but a misprinted numeral is the next lesson
foreach ($m in $heads) {
$v = Roman $m.Groups[1].Value
if ($v -ge $prev + 1 -and $v -le $prev + 3) { $keep += $m; $prev = $v; $nums += $v }
# a misprinted numeral is still near its place (XVV for XXV); a lesson's own small parts ("III. A GOOD NAME. (95)" inside lesson 12) are not
elseif ($m.Value -match '\(\d+\)\s*$' -and $v -ge $prev - 5) { $keep += $m; $prev++; $nums += $prev }
}
$heads = $keep
$parts = @($body.Substring(0, $heads[0].Index))
for ($i = 0; $i -lt $heads.Count; $i++) { $s = $heads[$i].Index + $heads[$i].Length; $e = if ($i + 1 -lt $heads.Count) { $heads[$i + 1].Index } else { $body.Length }; $parts += $body.Substring($s, $e - $s) }
}
# authors from the table of contents: "87. The Best Capital Louisa M. Alcott 245"
$tocAuthors = @{}
# (page column optional: the Fifth Reader's contents have none); the first entry for a number wins, which is the contents page
$tocText = $txt.Substring(0, $txt.LastIndexOf($(if ($cfg.start) { $cfg.start } else { 'x' })) + 1)
if ($cfg.tocEnd) { $ci = $tocText.IndexOf($(if ($cfg.tocStart) { $cfg.tocStart } else { 'CONTENTS' })); $tocText = $tocText.Substring($ci, $tocText.IndexOf($cfg.tocEnd, $ci) - $ci) } # only the contents page, not the chapters on reading aloud
foreach ($m in [regex]::Matches($tocText, '(?m)^\s*(\d+)\.\s+(\S.*?\S)\s{2,}(\S.*?\S)(?:\s{2,}(\d+))?\s*$')) {
$a = (($m.Groups[3].Value -split '\s{2,}')[-1] -replace '\.?\s*\d+$', '').TrimEnd('.') # "Longfellow.290": page number stuck to the name
if ($a -notmatch '^\d' -and -not $tocAuthors.ContainsKey([int]$m.Groups[1].Value)) { $tocAuthors[[int]$m.Groups[1].Value] = $a } # skip a line that holds two entries
}
$lessons = @()
for ($k = 0; $k -lt $heads.Count; $k++) {
$h = $heads[$k].Value.Trim()
$n = if ($h -eq 'Lesson 1') { 1 } else { Roman(([regex]::Match($h, 'LESSON ([IVXLC]+)')).Groups[1].Value) }
$headTitle = ''
if ($cfg.headRx) {
$hm = [regex]::Match($h, $cfg.headRx); $n = $nums[$k]
$headTitle = (Get-Culture).TextInfo.ToTitleCase($hm.Groups[2].Value.Trim().TrimEnd('.').Trim('"', [char]0x201c, [char]0x201d).TrimEnd('.').ToLower()) -creplace "(\w) N't", '$1n''t' -replace '\(Concluded\.?\)', '(concluded)' -replace '\(Continued\.?\)', '(continued)'
}
$review = $h -match 'REVIEW'
$block = $parts[$k + 1]
$block = $block -replace '\[Footnote:[^\]]*\]', '' -replace '(?m)^\s*(\d+|[IVX]+)\.\s*$', ''
# a title line glued to its poem ("ALICE'S SUPPER." then the verses) gets its own paragraph
$block = $block -creplace '(?m)^([A-Z][A-Z .,''\u2019!-]{3,})$\n(?=\S)', "`$1`n`n"
$author = ''; $defs = @(); $questions = @()
# DEFINITIONS / EXERCISES / NOTES at the end of a lesson (Fourth Reader)
$tm = [regex]::Match($block, '(?m)^[ \t]*(?:DEFINITIONS|EXERCISES|NOTES?)\b')
if ($tm.Success) {
$tail = $block.Substring($tm.Index); $block = $block.Substring(0, $tm.Index)
foreach ($sec in [regex]::Matches($tail, '(?s)(DEFINITIONS|EXERCISES|NOTES?)\.?\s*-*(.*?)(?=(?:\n[ \t]*(?:DEFINITIONS|EXERCISES|NOTES?)\b)|\z)')) {
$txt2 = ($sec.Groups[2].Value -replace '\s+', ' ').Trim()
if ($sec.Groups[1].Value -eq 'DEFINITIONS') {
$txt2 = $txt2 -replace '^In defining words, that meaning is given which is appropriate to them in the connection in which they are used\.\s*', ''
foreach ($ln in (($txt2 -replace '(?:^|\s)\d+\.\s+', "`n") -split "`n")) {
foreach ($entry in [regex]::Split($ln.Trim(), '(?<=\.)\s+(?=[A-Z][\w''/\-]*(?: [\w''/\-]+){0,2},\s)')) {
$cm = [regex]::Match($entry, '^([A-Z][\w''/\-]*(?: [\w''/\-]+){0,2}),\s+(.+?)\.?\s*$')
if ($cm.Success) {
$w = $cm.Groups[1].Value -replace '[''/\-]', ''
$d = ($cm.Groups[2].Value -replace '(?<=[a-z])-(?=[a-z])', '').Trim()
$defs += ,@($w, $d)
}
}
}
} elseif ($sec.Groups[1].Value -eq 'EXERCISES') {
$questions += @([regex]::Split($txt2, '(?<=\?)\s+') | ForEach-Object { $_.Trim() } | Where-Object { $_ -match '\?$' })
}
}
}
if ($cfg.prose) {
# the author's name under a poem: "John G. Whittier.", "Mrs. Follen."
$am = [regex]::Match($block, '\n[ \t]*((?:Mrs?\.\s+)?(?:[A-Z]\.\s*|[A-Z][a-z]+\s+){0,3}[A-Z][a-z]+\.)\s*$')
if ($am.Success) { $author = $am.Groups[1].Value.TrimEnd('.'); $block = $block.Substring(0, $am.Index) }
$block = $block -creplace '(?m)^(?=[ \t]*\d+\.\s+\S)', "`n" # each numbered paragraph is its own paragraph
foreach ($sp in ([regex]::Matches($block, '(?m)^([A-Z][a-z]+)\.\s+(?=["A-Z])') | Group-Object { $_.Groups[1].Value } | Where-Object { $_.Count -ge 2 -and $_.Name -notmatch '^(Mr|Mrs|Dr|St)$' })) {
$block = $block -creplace "(?m)^$($sp.Name)\.\s+", "`n$($sp.Name): " # a play: "Rose. See how..." -> "Rose: See how..."
}
}
# pull out illustrations (may span lines); script exercises become the cursive copy work
$script = @(); $picText = ''
foreach ($m in [regex]::Matches($block, '\[Illustration:([^\]]*)\]')) {
$c = $m.Groups[1].Value
if ($c -match '^\s*Script [Ee]xercise:') {
$c = $c -replace '^\s*Script [Ee]xercise:', ''
$paras = [regex]::Split($c.Trim(), '\n\s*\n')
foreach ($pp in $paras) {
if (IsPoem $pp) { $script += ($pp -split '\n' | ForEach-Object { $_.Trim() } | Where-Object { $_ }) } # poem: keep lines
else { $script += Sentences $pp }
}
} elseif (-not $picText) { $picText = ($c -replace '\s+', ' ').Trim().TrimEnd('.') }
}
$block = [regex]::Replace($block, '\[Illustration:[^\]]*\]', "`n`n@@PIC@@`n`n")
$paras = [regex]::Split($block.Trim(), '\n\s*\n') | ForEach-Object { $_.TrimEnd() } | Where-Object { $_.Trim() }
$words = @(); $letters = @(); $lines = @(); $inText = $false; $first = $true; $title = ''; $about = ''
foreach ($p0 in $paras) {
if ($p0.Trim() -eq '@@PIC@@') { if ($words.Count -or $letters.Count) { $inText = $true }; continue }
$p = $p0 -replace '\s*\([^)]*\)', ''
$isVocab = -not $cfg.prose -and -not $inText -and (($p -replace '\b(Mr|Mrs|Dr|St)\.', '$1') -notmatch '[.!?;,"]') -and ($first -or $p -match '\S\s{2,}\S' -or $p -match '^\S+$')
# phrase rows like "a rat a cat" come after the picture, not before it
if ($isVocab -and $words.Count -and $p -match '^\s*(a|the|an)\s\w+\s{3,}') { $isVocab = $false }
if ($isVocab) {
foreach ($ln in ($p0 -split '\n')) {
foreach ($t in (VocabTokens $ln)) {
if (-not $t) { continue }
$isSound = ($sounds -contains $t) -or ($t.Length -eq 1 -and -not ($first -and ($t -ceq 'a' -or $t -cmatch '^[A-Z]$')) -and $t -cne 'I')
if ($isSound) { if ($letters -notcontains $t) { $letters += $t } } elseif ($words -notcontains $t) { $words += $t }
}
$first = $false
}
continue
}
$inText = $true
# a note about the author before the selection ("By Celia Thaxter, who was born...", "Hester Lynch Thrale. 1739--1821, ...")
if ($cfg.headRx -and -not $lines.Count -and -not $about -and $p -match '\bborn\b|\d{4}\s*-{1,2}\s*\d{2,4}') {
$ta = if ($tocAuthors.ContainsKey([int]$n)) { ($tocAuthors[[int]$n] -split '\s+')[-1] } else { '' }
if ($p -match '^\s*By\s' -or ($ta -and $p -match "\b$([regex]::Escape($ta))\b")) { $about = ($p -replace '\s+', ' ').Trim(); continue }
}
if ($p.Trim() -cmatch '^[A-Z][A-Za-z'' ]{2,40}$' -and -not $lines.Count -and $p.Trim() -cmatch '^(?:(?:[A-Z][a-z'']*|a|an|and|the|of|in|on|at|to|for|with|by)\s?)+$' -and $p.Trim() -match '\s') { $title = $p.Trim(); continue }
if ($p.Trim() -cmatch '^[A-Z][A-Z .,''\u2019!?-]+$' -and ($cfg.prose -or $p -notmatch '\?')) { $title = (Get-Culture).TextInfo.ToTitleCase($p.Trim().TrimEnd('.').ToLower()); continue } # story title: THE WHITE KITTEN.
if (IsPoem $p) { $lines += ($p -split '\n' | ForEach-Object { ($_.Trim()) -replace '^\d+\.\s+', '' } | Where-Object { $_ }) } # poem
elseif ($p -notmatch '[.!?;,"]' -and $p -match '\S\s{3,}\S') { $lines += ($p.Trim() -split '\s{3,}') } # phrase row
else { $lines += Sentences $p }
}
if (-not $lines.Count) { $lines = $script }
$pic = ''; foreach ($key in $pics.Keys) { if ($picText -match "(?i)$key") { $pic = $pics[$key]; break } }
if ($cfg.pics.ContainsKey([int]$n)) { $pic = $cfg.pics[[int]$n] }
if (-not $pic -and $cfg.prose) { foreach ($key in $pics.Keys) { if ("$title $(($lines | Select-Object -First 3) -join ' ')" -match "(?i)\b$key(?:s|es)?\b") { $pic = $pics[$key]; break } } } # no picture notes: guess from the title
if (-not $pic) { $pic = if ($review -or $cfg.prose) { '📖' } else { '🖼️' } }
$o = [ordered]@{ n = $n; pic = $pic; picText = $(if ($picText) { $picText } elseif ($review) { 'Review' } else { '' }); words = @($words); letters = @($letters); lines = @($lines) }
if ($review) { $o.review = $true }
if ($headTitle) { $title = $headTitle } # the heading's title wins over part titles inside the lesson ("IV. WINTER.")
if ($defs.Count) {
# the book's own new words: shown in lowercase unless the lesson only ever writes them with a capital (names)
$alltext = $lines -join ' '
$o.defs = [ordered]@{}
foreach ($dd in $defs) {
$w = $dd[0]; if ($alltext -cmatch "\b$([regex]::Escape($w.ToLower()))\b" -or -not ($alltext -cmatch "\b$([regex]::Escape($w))\b")) { $w = $w.ToLower() }
if (-not $o.defs.Contains($w)) { $o.defs[$w] = $dd[1] }
}
$o.words = @($o.defs.Keys | Select-Object -First 12)
}
if ($questions.Count) { $o.questions = @($questions) }
if ($title) { $o.title = $title }
if ($cfg.headRx -and $tocAuthors.ContainsKey([int]$n)) { $author = $tocAuthors[[int]$n] } # the contents page knows best
if ($author -match 'Reader|EXAMPLES') { $author = '' }
if ($author) { $o.author = $author }
if ($about) { $o.about = $about }
if ($script.Count) { $o.script = @($script) }
if (-not $lines.Count) { Write-Host "Lesson $n has no text in Gutenberg's copy ($title), skipped" -ForegroundColor Yellow; continue }
$lessons += $o
}
if ($cfg.prose) {
# New words = words not met in the earlier books or earlier lessons, in reading order, up to 10 per lesson
$seen = New-Object 'System.Collections.Generic.HashSet[string]' ([StringComparer]::OrdinalIgnoreCase)
$wordRx = "[A-Za-z]+(?:'[A-Za-z]+)?"
foreach ($prev in $cfg.after) {
$pj = Get-Content "$root\public\books\$prev.json" -Raw -Encoding UTF8 | ConvertFrom-Json
foreach ($PL in $pj.lessons) { foreach ($s in @($PL.lines) + @($PL.words) + @($PL.script)) { foreach ($m in [regex]::Matches("$s", $wordRx)) { [void]$seen.Add($m.Value) } } }
}
foreach ($o in $lessons) {
$text = (@($o.lines) + @($o.script)) -join ' '
$new = @()
foreach ($m in [regex]::Matches($text, $wordRx)) {
$w = $m.Value
if ($w.Length -lt 2 -or $w -match "^n'" -or -not $seen.Add($w)) { continue }
# names keep their capital (capitalized in the middle of a line); other words are shown in lowercase
if (-not ($text -cmatch "(?:[a-z]|\b(?:Mr|Mrs|Miss|Dr)\.)\s+$([regex]::Escape($w))\b") -or ($text -cmatch "\b$([regex]::Escape($w.ToLower()))\b")) { $w = $w.ToLower() }
$new += $w
}
if (-not @($o.words).Count) { $o.words = @($new | Select-Object -First 10) } # the book's own word list wins when it has one
}
}
# Last clean-up of every line: printed page headers that ended up inside a sentence ("... ECLECTIC SERIES. ..."),
# footnote/section marks (#, *, [ ]), and scanned digits ("l0" = 10, "I1" = 11)
function CleanLine($s) {
$s = $s -creplace '\d*\s*\b(?:ECLECTIC (?:SERIES|READER)|(?:FIRST|SECOND|THIRD|FOURTH|FIFTH|SIXTH) READER)\b[.,]?\s*\d*', ' ' -replace '[#*\[\]]', ''
$s = $s -creplace '\bI1\b', 'I' -creplace '(?<=\d)l|l(?=\d)', '1' -creplace '^\d+\s+(?=[a-z])', '' -creplace '(?<=[a-z]{3})\d\b', '' # footnote number on a word ("orthography1"); "I1 will" = I; "l0" = 10; a page number before the text
($s -replace '\s{2,}', ' ').Trim()
}
foreach ($o in $lessons) {
$o.lines = @($o.lines | ForEach-Object { CleanLine $_ } | Where-Object { $_ -match '[A-Za-z]' })
if ($o.script) { $o.script = @($o.script | ForEach-Object { CleanLine $_ } | Where-Object { $_ -match '[A-Za-z]' }) }
}
$out = [ordered]@{ title = $cfg.title; sub = $cfg.sub; color = $cfg.color; source = "Project Gutenberg #$($cfg.gutenberg) (public domain)"; lessons = $lessons }
$json = $out | ConvertTo-Json -Depth 6
New-Item -ItemType Directory -Force "$root\public\books" | Out-Null
[IO.File]::WriteAllText("$root\public\books\$Book.json", $json, (New-Object Text.UTF8Encoding $false))
"$($lessons.Count) lessons"
$lessons | ForEach-Object { "{0,2} {1} w[{2}] l[{3}] lines={4} script={5} | {6}" -f $_.n, $_.pic, ($_.words -join ' '), ($_.letters -join ' '), $_.lines.Count, $(if ($_.script) { $_.script.Count } else { 0 }), $_.lines[0] }
EOFThe upload script is in the Second Reader post.
2. Make the lesson file
PC (PowerShell). It needs the first three books' lesson files (for the new words), prints the two skipped lessons in yellow, then "77 lessons" and one line per lesson:
powershell -ExecutionPolicy Bypass -File C:\ABCBooks\tools\make-book.ps1 -Book third18793. Record it with Eloquence
PC (PowerShell). About 16,600 recordings, about 10 minutes here:
powershell -ExecutionPolicy Bypass -File C:\ABCBooks\tools\make-audio.ps1 -Book third1879Send them to CT 306 (abc-web) and turn them into MP3s, 12 batches, about 10 minutes:
powershell -ExecutionPolicy Bypass -File C:\ABCBooks\tools\push-audio.ps14. Put it on the server
PC (PowerShell):
powershell -ExecutionPolicy Bypass -File C:\ABCBooks\deploy.ps1Open https://games.503mail.com/abcbooks/, pick a name, and the purple McGuffey's Third Eclectic Reader opens its lesson map.
Troubleshooting
The logs and the Tablet check page are the same as in the Primer post; the disk checks for CT 306 are in the Second Reader post. For this book:
"/abcbooks/books/third1879.json","/abcbooks/audio/third1879.json" | % { (Invoke-WebRequest "https://games.503mail.com$_" -UseBasicParsing).StatusCode }- Look for: 200 twice (PC, PowerShell). A 404 means a step above was skipped.
Notes
- Disk: CT 306 (abc-web) has 8 GB; all 8 books together use about 2.7 GB.
- Still to come: the Fourth, Fifth and Sixth Readers (each has its own layout again: the Fourth has definitions with every lesson, the Fifth and Sixth open with long chapters on reading aloud), then Spell & Read from the 1867 Pictorial Primer.