Discussion:
[slrn] Unable to score articles based on Xref: or Newsgroups: headers.
Add Reply
Lafe
2021-08-25 16:09:06 UTC
Reply
Permalink
Hello folks,

I'm a bit new to slrn, and I've been having this scoring issue for a bit, and
haven't been able to figure out where I'm going wrong.

Any scores that I apply to the Xref: or Newsgroups: headers don't seem to work
for me at all.

I was using version 1.0.3 on ubuntu 20.04. I saw that there were developer
preview versions available, so I updated to the latest version and built from
source. But this didn't contain a fix for my issue.
slrn pre1.0.4-5
* Note: This version is a developer preview.
S-Lang Library Version: pre2.3.3-58
Operating System: Linux
Backends: +nntp -slrnpull -spool
External programs / libs: -canlock -inews +ssl -uudeview +iconv
Features: +decoding +emphasized_text +end_of_thread +fake_refs +gen_msgid
-grouplens -msgid_cache +piping +rnlock +spoilers -strict_from
Using 64 bit integers for article numbers.
Default server object: nntp
Default posting mechanism: nntp
Below is what's currently in my score file, I added the Google and killfile
scores just to see if scoring was working at all, and it is, both of the other
two scores work and I'm able to see that articles are killed and/or scored by
them. Only the first one does nothing, and I've tried a number of different
variations... but this Xref: rule pulled directly from the examples doesn't have
any effect.

I have also tried it with and without prefer_head. I can bring up articles and
see that there are multiple colons in the Xref: header as displayed by slrn.

Any ideas? Thanks so much for any help!

score file:
---------------------

[*]
Score: -100
Xref: :.*:

Score: -50
From: Google

Score: =-9999
From: test-***@nospam\.example
Tavis Ormandy
2021-08-26 16:39:30 UTC
Reply
Permalink
Post by Lafe
I have also tried it with and without prefer_head. I can bring up articles and
see that there are multiple colons in the Xref: header as displayed by slrn.
Any ideas? Thanks so much for any help!
---------------------
[*]
Score: -100
It looks okay to me, have you looked at the view_scores dialog on an
article you think it should match on? (Default key is 'v'), maybe the
score is being cancelled out...

Tavis.
--
_o) $ lynx lock.cmpxchg8b.com
/\\ _o) _o) $ finger ***@sdf.org
_\_V _( ) _( ) @taviso
Lafe
2021-08-26 17:26:54 UTC
Reply
Permalink
Post by Tavis Ormandy
Post by Lafe
---------------------
[*]
Score: -100
It looks okay to me, have you looked at the view_scores dialog on an
article you think it should match on? (Default key is 'v'), maybe the
score is being cancelled out...
Thanks for looking! I didn't know about that view_scores, but I tried it on an
article in this newsgroup that had an Xref header that contained two colon ":"
characters in its contents, and it told me that the article didn't match on any
rules.

I'm guessing that's the root of the problem. It doesn't seem to like that
specific regex, or just isn't able to apply it to that header correctly.

You say that it does work for you though? I'm not sure what could be
different.

If I add a Newsgroups: rule, and then attempt open a newsgroup, it appears that
slrn will download articles automatically to be able to check the rule. But
only sometimes. And if it attempts this against a newsgroup with more than
about 20 unread articles it says that there's an error and disconnects me and
quits/crashes to the command line. That may be something weird with my nntp
provider though (newsdemon).

Scoring on Xref or Newsgroups headers isn't a deal breaker, it's just strange
and I wish I could figure out why it doesn't work.
Tavis Ormandy
2021-08-26 17:49:15 UTC
Reply
Permalink
Post by Lafe
Thanks for looking! I didn't know about that view_scores, but I tried it on an
article in this newsgroup that had an Xref header that contained two colon ":"
characters in its contents, and it told me that the article didn't match on any
rules.
I'm guessing that's the root of the problem. It doesn't seem to like that
specific regex, or just isn't able to apply it to that header correctly.
You say that it does work for you though? I'm not sure what could be
different.
Yes, I use rules like this that work fine:

Score: -10 % X-Posted > 3
Xref: :.*:.*:.*:
Score: -10 % X-Posted > 4
Xref: :.*:.*:.*:.*:
Score: -10 % X-Posted > 5
Post by Lafe
If I add a Newsgroups: rule, and then attempt open a newsgroup, it appears that
slrn will download articles automatically to be able to check the rule. But
only sometimes. And if it attempts this against a newsgroup with more than
about 20 unread articles it says that there's an error and disconnects me and
quits/crashes to the command line. That may be something weird with my nntp
provider though (newsdemon).
Hmm, maybe try `set prefer_head 2`? If that fixes it, seems like it is
your provider (Note that scoring will be much slower without overviews,
but maybe that doesn't bother you!).
Post by Lafe
Scoring on Xref or Newsgroups headers isn't a deal breaker, it's just strange
and I wish I could figure out why it doesn't work.
Maybe also verify there is no accidental weird line endings or
something? You could create a fresh scorefile like:

$ printf "Score: -10\nXref: :.*:\n" > scorefile

Tavis.
--
_o) $ lynx lock.cmpxchg8b.com
/\\ _o) _o) $ finger ***@sdf.org
_\_V _( ) _( ) @taviso
Lafe
2021-08-26 19:07:00 UTC
Reply
Permalink
Post by Tavis Ormandy
Hmm, maybe try `set prefer_head 2`? If that fixes it, seems like it is
your provider (Note that scoring will be much slower without overviews,
but maybe that doesn't bother you!).
Thanks for the suggestion! Setting it to prefer_head did fix it, so I agree that
seems to point to an issue with my nntp server. It changed to the behavior I see
with "Newsgroups:" rules, where it wants to download all of the articles to
perform the scoring.

So now if I pull more than 15 or 20 articles when I enter a group, slrn says
there's an error and "crashes" to the command line saying that it can't recover.
I think this is also likely a weirdness with how it requests the articles and my
news provider killing the connection or something.

I appreciate your help in the troubleshooting! I can live without cumulative
scoring on crossposts while I'm using this nntp provider.
Eli the Bearded
2021-08-26 21:42:39 UTC
Reply
Permalink
Post by Lafe
Post by Tavis Ormandy
Hmm, maybe try `set prefer_head 2`? If that fixes it, seems like it is
your provider (Note that scoring will be much slower without overviews,
but maybe that doesn't bother you!).
Thanks for the suggestion! Setting it to prefer_head did fix it, so I
agree that seems to point to an issue with my nntp server. It changed
to the behavior I see with "Newsgroups:" rules, where it wants to
download all of the articles to perform the scoring.
Have you tried to look at the overview records yourself?

Path: [...]
+ feed.usenet.farm!feed.usenet.farm!tr2.eu1.usenetexpress.com!feeder.
+ usenetexpress.com!tr1.iad1.usenetexpress.com!
+ 2a07:8080:119:fe:f19b:d5b2:4545:7979.MISMATCH!news.newsdemon.com!not-
+ for-mail

Newsdemon seems to be using IPv6 without IPv6 reverse names.

Anyway, general method is:

telnet news.newsdemon.com 119
AUTHINFO USER username
AUTHINFO PASS password
LIST overview.fmt
XOVER group.name:articlenum
QUIT

AUTHINFO only used if needed, there's also an "AUTHINFO GENERIC ..."
variation. The overview.fmt file gives the details of what to expect.
The local one here is:

Subject:
From:
Date:
Message-ID:
References:
Bytes:
Lines:
Xref:full
NNTP-Posting-Host:full

That gives the meaning of each line in the output, and the ones marked
"full" will have a header name in front of them, the overs just the
field value.

The XOVER command can get a single article or a range.
XOVER news.software.readers:1000
XOVER news.software.readers:1000-1050

I've heard of bugs in newsreaders with values after the standard
Subject: to Lines: entries that _don't_ have the header name. One
such bug is why this site has "NNTP-Posting-Host:full". I think
it was in Xnews.
Post by Lafe
So now if I pull more than 15 or 20 articles when I enter a group,
slrn says there's an error and "crashes" to the command line saying
that it can't recover.
I think this is also likely a weirdness with how it requests the
articles and my news provider killing the connection or something.
As someone who never uses slrn, but has a vague interest in nntp
clients, I'm curious what's going on here. I looked into this thread
because I suspected some overview db issues.

Elijah
------
uses trn and perl NNTP module scripts for his news
Lafe
2021-09-15 19:14:47 UTC
Reply
Permalink
Post by Eli the Bearded
Have you tried to look at the overview records yourself?
Path: [...]
+
feed.usenet.farm!feed.usenet.farm!tr2.eu1.usenetexpress.com!feeder. +
usenetexpress.com!tr1.iad1.usenetexpress.com! +
2a07:8080:119:fe:f19b:d5b2:4545:7979.MISMATCH!news.newsdemon.com!not-
+ for-mail
Newsdemon seems to be using IPv6 without IPv6 reverse names.
telnet news.newsdemon.com 119
AUTHINFO USER username
AUTHINFO PASS password
LIST overview.fmt
XOVER group.name:articlenum
QUIT
AUTHINFO only used if needed, there's also an "AUTHINFO GENERIC ..."
variation. The overview.fmt file gives the details of what to expect.
Xref:full
NNTP-Posting-Host:full
That gives the meaning of each line in the output, and the ones marked
"full" will have a header name in front of them, the overs just the
field value.
The XOVER command can get a single article or a range.
XOVER news.software.readers:1000
XOVER news.software.readers:1000-1050
I've heard of bugs in newsreaders with values after the standard
Subject: to Lines: entries that _don't_ have the header name. One
such bug is why this site has "NNTP-Posting-Host:full". I think
it was in Xnews.
Post by Lafe
So now if I pull more than 15 or 20 articles when I enter a group,
slrn says there's an error and "crashes" to the command line saying
that it can't recover.
I think this is also likely a weirdness with how it requests the
articles and my news provider killing the connection or something.
As someone who never uses slrn, but has a vague interest in nntp
clients, I'm curious what's going on here. I looked into this thread
because I suspected some overview db issues.
Elijah
------
uses trn and perl NNTP module scripts for his news
I'm sorry that I somehow missed your follow-up to my post before. I tried
your suggestion, and I had the following results:

~$ telnet news.newsdemon.com 119
Trying 185.151.13.247...
Connected to nd.tlsusenet.com.
Escape character is '^]'.
200 Welcome (n14.iad1)
AUTHINFO USER **************
381 Proceed with PASS
AUTHINFO PASS **************
281 Hello **************!
LIST overview.fmt
215 Information follows (multi-line)
Subject:
From:
Date:
Message-ID:
References:
Bytes:
Lines:
Xref:full
.
XOVER news.software.readers:1000
400 Unrecognized command
OVER eli$***@qaz.wtf
400 Unrecognized command
OVER news.software.readers:1000
400 Unrecognized command
QUIT
205 Goodbye
Connection closed by foreign host.

I'm guessing the complete lack of support for XOVER/OVER is what was
causing this issue with SLRN, and why prefer-head 2 "sort of" worked.

Thanks for the suggestion!

Lafe

Loading...