The br is not your friend in extracted PDF text

TLDR

Joining multi-line sentence fragments with HTML <br> tags to preserve PDF line breaks creates fragmented, un-reflowable HTML. PDF line breaks are physical typesetting artifacts of fixed page width, not semantic paragraph breaks. Replacing <br> joins with whitespace produces clean, continuous paragraph text that adapts naturally across responsive viewports and contenteditable editors.

Paragraph Joining Strategy	Generated HTML Structure	Plain-Text Export	Responsive Viewport Behavior
Visual <br> Joining	<p>Text fragment<br>next line...</p>	Preserves hard wraps & hanging hyphens	Breaks layout on mobile viewports
Semantic Space Joining	<p>Text fragment next line...</p>	Clean, continuous sentences	100% Responsive Text Reflow

---

Problem statement: the <br> joining defect

PDF text extraction engines output text items with exact X/Y coordinates. Lines belonging to a single paragraph are emitted as separate physical strings.

Our early sentence-aware paragraph joiner inserted <br> tags between consecutive lines within a single paragraph:

// DEFECTIVE IMPLEMENTATION: Preserving visual line breaks via <br>
function joinParagraphLines(lineItems) {
  return lineItems.reduce((acc, line) => {
    return acc ? ${acc}&lt;br&gt;${line.text} : line.text;
  }, '');
}

This produced HTML content like:

<p>Unexpectedly,<br>such degradation is not caused by overfitting, and adding<br>more layers to a suitably deep model leads to higher train-<br>ing error, as reported in [11, 42].</p>

---

Technical failure mode: conflating layout artifacts with semantics

1. Broken Responsive Reflow: Fixed <br> tags prevent text from reflowing when users resize viewports or edit text in responsive UI wrappers.
2. Hanging Line-Break Hyphens: PDF typesetting engines insert hyphens at line breaks (train-\ning). Joining with <br> leaves trailing hyphens floating mid-sentence.
3. DOM Inspector Noise: Every <br> creates an explicit DOM node that interferes with rich-text editor cursors and selection ranges.

---

The fix & architecture: semantic space joining with dehyphenation

Replace <br> joins with single spaces, and run a regex pass to repair line-break hyphens:

// REFACTORED: Semantic Paragraph Joining & Dehyphenation
export function assembleParagraphText(lineItems) {
  let combinedText = lineItems.map(item => item.text.trim()).join(' ');
// Dehyphenate words broken across visual line wraps   // Example: "train- ing" -> "training"   combinedText = combinedText.replace(/(\w+)-\s+([a-z]+)/gi, '$1$2');
return <p>${escapeHtml(combinedText)}</p>; }

The refactored output yields clean, continuous HTML:

<p>Unexpectedly, such degradation is not caused by overfitting, and adding more layers to a suitably deep model leads to higher training error, as reported in [11, 42].</p>

Rule of thumb: Treat visual line breaks in extracted PDF text as layout artifacts rather than semantic boundaries. Join multi-line paragraph text with spaces and apply dehyphenation passes.