-
Notifications
You must be signed in to change notification settings - Fork 0
/
concept.html
124 lines (119 loc) · 5.84 KB
/
concept.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="language" content="EN">
<title>R. S. Doiel, Software Engineer/Analyst - concept</title>
<link rel="stylesheet" type="text/css" href="/printfonts/print.css" media="print" />
<link rel="stylesheet" type="text/css" href="/webfonts/fonts.css" media="screen" />
<link rel="stylesheet" type="text/css" href="/css/site.css" media="screen" />
<link title="RSS feed for rsdoiel's blog" rel="alternate" type="application/rss+xml" href="https://rsdoiel.github.io/rss.xml" />
<link title="markdown source for page" rel="alternative" type="application/markdown" href="concept.md">
</head>
<body>
<nav>
<ul>
<li><a href="/">Home</a></li>
<li><a href="index.html">README</a></li>
<li><a href="user-manual.html">User Manual</a></li>
<li><a href="LICENSE">LICENSE</a></li>
<li><a href="INSTALL.html">Install</a></li>
<li><a href="search.html">Project Search</a></li>
<li><a href="ideas.html">Someday, Maybe</a></li>
<li><a href="https://github.com/rsdoiel/stngo">GitHub</a></li>
<li><a href="about.html">About</a></li>
</ul>
</nav>
<section>
<!-- <h1>concept</h1> -->
<h1 id="skimmer">skimmer</h1>
<p>By R. S. Doiel, 2023-10-06</p>
<p>I have a problem. I like to read my feeds in newsboat but I can’t
seem to get it working on a few machines I use. I miss having access to
read feeds. Additionally there are times I would like to read my feeds
in the same way I read twtxt feeds using
<code>yarnc timeline | less -R</code>. Just get a list of all items in
reverse chronological order.</p>
<p>I am not interested in reinventing newsboat, it does a really good
job, but I do want an option where newsboat isn’t available or is not
not convenient to use. This lead me to think about an experiment I am
calling skimmer . Something that works with RSS, Atom and jsonfeeds in
the same way I use <code>yarnc timeline | less -R</code>.<br />
I’m also inspired by Dave Winer’s a river of news site and his outline
tooling. But in this case I don’t want an output style output, just a
simple list of items in reverse chronological order. I’m thinking of a
more ephemeral experience in reading.</p>
<p>This has left me with some questions.</p>
<ul>
<li>How simple is would it be to write skimmer?</li>
<li>How much effort would be required to maintain it?</li>
<li>Could this tool incorporate support for other feed types,
e.g. twtxt, Gopher, Gemini?</li>
</ul>
<p>There is a Go package called <a
href="https://github.com/mmcdole/gofeed">gofeed</a>. The README
describes it as a “universal” feed reading parser. That seems like a
good starting point and picking a very narrowly focus task seems like a
way to keep the experiment simple to implement.</p>
<h2 id="design-issues">Design issues</h2>
<p>The reader tool needs to output to standard out in the same manner as
<code>yarnc timeline</code> does. The goal isn’t to be newsboat, or
river of news, drummer, or Lynx but to present a stream of items
usefully formatted to read from standard output.</p>
<p>Some design ideas</p>
<ol type="1">
<li>Feeds should be fetched by the same tool as the reader but that
should be done explicitly (downloads can take a while)</li>
<li>I want to honor that RSS does not require titles! I need to handle
that case gracefully</li>
<li>For a given list of feed URLs I want to save the content in a
SQLite3 database (useful down the road)</li>
<li>I’d like the simplicity of newsboat’s URL list but I want to
eventually support OPML import/export</li>
</ol>
<h1 id="skimmer-a-thin-wrapper-around-gofeed">Skimmer, a thin wrapper
around gofeed</h1>
<p>In terms of working with RSS, Atom and JSON feeds the <a
href="https://github.com/mmcdole/gofeed">gofeed</a> takes care of all
the heavy lifting in parsing that content. The go http package provides
a reliable client. There is a pure Go package, <a href="">go-sqlite</a>,
for integrating with SQLite 3 database. The real task is knitting this
together and a convenient package.</p>
<p>Here’s some ideas about behavior.</p>
<p>To configure skimmer you just run the command. It’ll create a
directory at <code>$HOME/.skimmer</code> to store configuration much
like newsboat does with <code>$HOME/.newsboat</code>.</p>
<pre><code>skimmer</code></pre>
<p>A default URL list to be created so when running the command you have
something to fetch and read.</p>
<p>Since fetching feed content can be slow (this is true of all news
readers I’ve used) I think you should have to explicitly say fetch.</p>
<pre><code>skimmer -fetch</code></pre>
<p>This would read the URLs in the URL list and populate a simple SQLite
3 database table. Then running skimmer again would display any harvested
content (or running skimmer in another terminal session).</p>
<p>Since we’re accumulating data in a database there are some house keep
chores like prune that need to be supported. Initial this can be very
simple and if the experiment move forward I can improve them over time.
I want something like saying prune everything up to today.</p>
<pre><code>skimmer -prune today</code></pre>
<p>There are times I just want to limit the number of items displayed so
a limit options makes sense</p>
<pre><code>skimmer -limit 10</code></pre>
<p>Since I am displaying to standard out I should be able to output via
Pandoc to pretty print the content.</p>
<pre><code>skimmer -limit 50 | pandoc -t markdown -f plain | less -R</code></pre>
<p>That seems a like a good set of design features for an initial
experiment.</p>
<h2 id="proof-of-concept-implementation">Proof of concept
implementation</h2>
<p>Spending a little time this evening. I’ve release a proof of concept
on GitHub at <a href="https://github.com/rsdoiel/skimmer"
class="uri">https://github.com/rsdoiel/skimmer</a>, you can read the
initial documentation at <a
href="https://rsdoiel.github.io/skimmer">skimmer</a>.</p>
</section>
<footer>
</footer>
</body>
</html>