Merge branch 'release/0.2'

Author: tanhaow

.github/ISSUE_TEMPLATE/software_release.md

@@ -13,26 +13,33 @@ assignees: ''
 
 ### prep release candidate for acceptance testing
 
-- [ ] Update the version number to the appropriate release candidate number (e.g., `0.5rc1`)
+- [ ] Update the version number in `src/remarx/__init__.py` to the appropriate release candidate number (e.g., `0.5rc1`)
 - [ ] Create a draft PR (since it should not be merged)
 - [ ] Review the changelog to make sure that all features, changes, bugfixes, etc included in the release are documented. You may want to review the git revision history to be sure you've captured everything.
 - [ ] Review the README to make sure that its contents are up to date
 - [ ] Check python requirements for any internal dependencies that should be released (or at least pinned to a specific git commit)
 - [ ] Confirm that all checks for the draft PR pass (e.g., unit tests, code coverage checks)
-- [ ] Review code documentation to make sure it is up to date.
+- [ ] Build documentation on the release branch and run the server to review and make sure it is up to date.
 
-### acceptance testing fails
+### BEFORE acceptance testing
+
+- [ ] Review issues included in the release to make sure they have testing instructions before marking them as ready for acceptance testing.
+- [ ] Give project team members instructions about how to install the release candidate version.
+
+### IF acceptance testing fails
+
+*These steps are only needed if acceptance testing fails and you need to update and retest the release candidate.*
 
 - [ ] Increment the version number to the next release candidate (e.g., `0.5rc1` → `0.5rc2`)
 - [ ] Address the changes raised in acceptance testing and repeat the previous section.
 
-### acceptance testing passes
+### WHEN acceptance testing passes
 
 - [ ] Set the final release version number (e.g., `0.5rc1` → `0.5`)
 - [ ] Use git flow to finish the release (merge release branch into both main and develop, create a tag, remove the release branch, etc.). (`git flow release finish 0.5`)
 
 ## after release
 
-- [ ] Increase the develop branch version so it is set to the next expected release (i.e., if you just released `0.5` then develop will probably be `0.6-dev` unless you are working on a major update, in which case it will be `1.0-dev`)
+- [ ] Increase the develop branch version so it is set to the next expected release (i.e., if you just released `0.5` then develop will probably be `0.6.dev0` unless you are working on a major update, in which case it will be `1.0.dev0`)
 - [ ] Push all updates to GitHub (main branch, develop branch, tags)
 - [ ] Create a GitHub release for the new tag, to trigger package publication on PyPI (and eventually Zenodo DOI)

.github/workflows/check.yml

@@ -27,3 +27,5 @@ jobs:
         # Note: default deps currently include ruff, but probably should not
       - name: Check with ruff
         run: uv run ruff check
+      - name: Check formatting with ruff
+        run: uv run ruff format --check

.pre-commit-config.yaml

@@ -25,6 +25,7 @@ repos:
         additional_dependencies:
           - mdformat-mkdocs
           - mdformat-frontmatter
+          - mdformat-pyproject # support configuration in pyproject.toml
         args: [--wrap, keep]
         # Temporarily exclude docs/api/index.md due because it messes up the formatting
         exclude: ^docs/api/index\.md$
@@ -34,6 +35,7 @@ repos:
     hooks:
       - id: codespell
         exclude_types: ["xml"] # ignore sample xml file
+        args: [--uri-ignore-words-list=archiv] # ignore German "archiv" in URLs
   # Common file checks (no dependencies)
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v4.5.0
@@ -51,3 +53,8 @@ repos:
     rev: 0.8.6
     hooks:
       - id: uv-lock
+  # validate GitHub Actions workflow files
+  - repo: https://github.com/mpalmer/action-validator
+    rev: v0.8.0
+    hooks:
+      - id: action-validator

CHANGELOG.md

@@ -1,5 +1,35 @@
 # CHANGELOG
 
+## 0.2.0
+
+### Application
+
+- The app now consists of two notebooks (Sentence Corpus Builder & Quote Finder)
+- Logging is now automatically configured by the application, and the log file location is reported to the user
+- Quote Finder notebook now supports quotation detection between two sentence corpus files (original and reuse)
+
+### Documentation
+
+- Add technical design document to MkDocs documentation
+
+### Sentence corpus creation
+
+- Add sentence id field (`sent_id`) to generated sentence corpora
+- Processes TEI/XML documents to yield separate chunks for body text and footnotes, with each footnote yielded individually as a separate element
+
+### Quotation detection
+
+- Add a method for generating sentence embeddings from a list of sentences
+- Added method for identifying likely quote sentence pairs
+
+### Scripts
+
+- Add `parse_html` script for converting the manifesto html files to plain text for sentence corpus input (one-time use)
+
+### Misc
+
+- Add a utility method (`configure_logging`) to configure logging, supporting logging to a file or to stdout
+
 ## [0.1.0] - 2025-09-08
 
 _Initial release._
@@ -26,7 +56,6 @@ _Initial release._
 
 ### Misc
 
-- Add GitHub Actions workflow to build and publish python package on PyPI when a new GitHub release
- created
+- Add GitHub Actions workflow to build and publish python package on PyPI when a new GitHub release created
 
 [0.1.0]: https://github.com/Princeton-CDH/remarx/tree/0.1

DEVELOPERNOTES.md

@@ -53,9 +53,9 @@ changes should not be included in the changelog unless it is substantial enough
     - Via GitHub web interface: Go to your PR → Labels section in right sidebar → Click gear icon → Type "no changelog" and select it
     - Via GitHub CLI: Run `gh pr edit --add-label "no changelog"`
 
-1. The changelog check will automatically re-run and pass when the label is applied
+2. The changelog check will automatically re-run and pass when the label is applied
 
-1. Remove the label if you later decide the PR does need a changelog entry
+3. Remove the label if you later decide the PR does need a changelog entry
 
 ## Documentation
 

README.md

@@ -51,7 +51,7 @@ The app will not close automatically when you close the browser window or tab.
 To close the app:
 
 1. Type control+c within the terminal where the `remarx-app` command was run
-1. Then, when prompted, type `y` followed by enter.
+2. Then, when prompted, type `y` followed by enter.
 
 ## Documentation
 

docs/api/index.md

@@ -5,9 +5,11 @@ hide: [navigation]
 
 # ::: remarx
 
+## ::: remarx.utils
+
 ## ::: remarx.app
 
-## ::: remarx.app_utils
+## ::: remarx.app.utils
 
 ## ::: remarx.sentence
 
@@ -26,3 +28,9 @@ hide: [navigation]
 #### ::: remarx.sentence.corpus.text_input
     options:
       members: false
+
+## ::: remarx.quotation
+
+### ::: remarx.quotation.embeddings
+
+### ::: remarx.quotation.pairs

docs/images/system-architecture-diagram.md

@@ -0,0 +1,45 @@
+# System Architecture Diagram
+
+```mermaid
+
+%%{init: {
+  "theme": "base",
+  "themeVariables": { "fontFamily": "Inter, ui-sans-serif, system-ui" },
+  "flowchart": { "curve": "basis", "nodeSpacing": 40, "rankSpacing": 60 }
+}}%%
+flowchart LR
+  classDef source  fill:#f3f4f6,stroke:#475569,stroke-width:1px,rx:6,ry:6;
+  classDef builder fill:#00d5e0,stroke:#0891b2,stroke-width:2px,stroke-dasharray:4 3,rx:14,ry:14,color:#00323a;
+  classDef process fill:#e6eefc,stroke:#6b8bd6,stroke-width:1.5px,rx:12,ry:12;
+  classDef corpus  fill:#ffffff,stroke:#64748b,stroke-width:1px,rx:8,ry:8;
+  classDef output  fill:#ffffff,stroke:#111827,stroke-width:1.5px,rx:8,ry:8;
+  classDef invis   fill:transparent,stroke:transparent;
+
+
+  B["Communist<br/>Manifesto<br/>(HTML)"]:::source --> E
+  C["DNZ<br/>Texts<br/>(XML)"]:::source --> E
+  A["MEGAdigital<br/>Texts<br/>(XML)"]:::source --> E
+  D["Other<br/>Plaintext<br/>(TXT)"]:::source -.-> E
+
+  E["sentence<br/>corpus<br/>builder"]:::builder
+  E --> F["Original<br/>Sentences<br/>(CSV)"]:::corpus
+  E --> G["Reuse<br/>Sentences<br/>(CSV)"]:::corpus
+
+  F -.-> K
+
+  subgraph CP["core pipeline"]
+    direction LR
+    I["Sentence-Level<br/>Quote Detection"]:::process
+    J["Quote<br/>Sentence Pairs<br/>(CSV)"]:::corpus
+    K["Quote<br/>Compilation"]:::process
+    L["Quotes<br/>Corpus<br/>(CSV)"]:::output
+    I --> J --> K --> L
+  end
+  style CP fill:#c8f7ff,stroke:#0891b2,stroke-width:2px,rx:18,ry:18;
+
+  F --> I
+  G --> I
+
+  G -.-> K
+
+```