I think the one point default per comment isn't added? It's only additional points? At least, that's what I've noticed

There is something more than that. I noticed that the few highly-voted submissions I have didn't add the whole amount to my karma.

I have the impression that submission votes only count half of what comment votes count, or something along those lines.

Earlier upvotes are more impactful in getting a post on the front page and in affecting its rank (because the rank algorithm uses time decay), but they don't count any differently towards karma.

Is there really no delay, or anything else going on? I'm sure I've seen my upvote count be out of sync with the changes I see on my scores for recent comments.

Does https://news.ycombinator.com/item?id=26868069 answer your question?

I could be wrong but I think it used to be 1:1 with submission upvotes. It may still be linear but less than 1:1, I'm not sure. It is still 1:1 with comment upvotes I believe.

That makes sense to me, a user submitting a press release or news article about a current event being the "lucky winner" where their submission gets traction doesn't automatically equate to them being a good community member in general.

I think some sites are weighted differently. More commonly posted sites require multiple votes for one point added. Less commonly posted sites require less upvotes for a point added.

Or something like that anyway.

It's 1:1 with comment upvotes, less with submission upvotes (how much less I'm not sure but my observation would peg it around 3:1). Personally I think it should be the other way around as a submission can add a lot more value than a comment.

It's not 1-1 with either comment or submission upvotes. It's more complicated than that, largely for anti-abuse reasons.

I have noticed that a single downvote sometimes counts for -2 rather than -1. Sometimes I'll undo a downvote because I think a comment wasn't bad enough to cost 2 points. I have been trying to spot the pattern behind it for over a year with no luck. Can you explain this or is it secret?

Hah well my two downvotes here and two corresponding lost points say otherwise. But I'll take your word for it.

In a weaker gravity field with a low sample frequency the algorithm does indeed appear 1:1. The complexity arises once a peripheral second gravity field can no longer be ignored or a certain noise threshold is passed.

The equation ends up looking something like this: https://slack-files.com/TEGQB8GNS-F01UU5FMKRR-820b186081

Once you start thinking about it in terms of the definite integrals of a piecewise function you'll see where the weights would be applied in more extreme situations.