remove pkg/context, overhaul snapshot handling

[jaypipes]

Gets rid of pkg/context entirely along with the
Do()/Setup()/Teardown() context-handling stuff that was added to deal with snapshots. Moves to a simpler and more straight-forward system of passing zero or more Option functors that build an Options struct.

Removes the ghw-snapshot tool and makes the ghwc tool understand ghw snapshots natively (just pass a -s <snapshot_path> CLI flag). Added a ghwc snapshot command that creates snapshots.

[ffromani]

First of all thanks for filing this! design wise I like this approach a lot and I'm all for it.
The main question I have is: how do we deal with backward compatibilty? this is an API breakage both at golang package level and at command line level (tool removal).
Perhaps we can bump the major version? would that too much of a jump, or too little?

Glancing over the PR everything seems reasonnable, but it will take me some time to ready everything (other duties :\ )

[jaypipes]

First of all thanks for filing this! design wise I like this approach a lot and I'm all for it. The main question I have is: how do we deal with backward compatibilty? this is an API breakage both at golang package level and at command line level (tool removal). Perhaps we can bump the major version? would that too much of a jump, or too little?

Glancing over the PR everything seems reasonnable, but it will take me some time to ready everything (other duties :\ )

@ffromani Ciao! :) Yeah, it's a breaking change for sure, though I worked hard to minimize the changes that "normal" users would see. The call signature for all of the Info() functions remains the same so their Go code that imports the ghw library would not need to change. And the ghwc CLI tool hasn't changed its command signatures -- only added a -s <snapshot path> option to allow reading snapshots). That said, for "power" users that used the ghw-snapshot tool, obviously that's changed quite a bit.

I don't think it warrants a major version bump because there's no functional change to the API besides the removal of the SnapshotOptions/WithSnapshot functor. But, again, I don't think many people outside of us ever used that feature of the API/low-level calling interface.

I did deprecate the top-level (alias.go) WithOption and renamed it just Option, but that's not a breaking change due to continuing to keep the old WithOption type in place.

So, I think (unless you say otherwise) this should be "safe" to cut as another 0.x release. :)

[ffromani]

Note IF we decide we bump the major version, we can and probably should revamp and conclude my ancient SRIOV PR. It's stuck because last time I worked on it I could not find a reasonnably high backward compatibility.

[ffromani]

I don't think it warrants a major version bump because there's no functional change to the API besides the removal of the SnapshotOptions/WithSnapshot functor. But, again, I don't think many people outside of us ever used that feature of the API/low-level calling interface.

Sound reasonnable. I didn't yet grasp this because I only had a cursor glance over your PR, which I did because I wanted to acknowledge this work and the review request. I will have a deep enough look ASAP and comment again.

[ffromani]

First of all thanks for filing this! design wise I like this approach a lot and I'm all for it. The main question I have is: how do we deal with backward compatibilty? this is an API breakage both at golang package level and at command line level (tool removal). Perhaps we can bump the major version? would that too much of a jump, or too little?
Glancing over the PR everything seems reasonnable, but it will take me some time to ready everything (other duties :\ )

@ffromani Ciao! :) Yeah, it's a breaking change for sure, though I worked hard to minimize the changes that "normal" users would see. The call signature for all of the Info() functions remains the same so their Go code that imports the ghw library would not need to change. And the ghwc CLI tool hasn't changed its command signatures -- only added a -s <snapshot path> option to allow reading snapshots). That said, for "power" users that used the ghw-snapshot tool, obviously that's changed quite a bit.

I don't think it warrants a major version bump because there's no functional change to the API besides the removal of the SnapshotOptions/WithSnapshot functor. But, again, I don't think many people outside of us ever used that feature of the API/low-level calling interface.

I did deprecate the top-level (alias.go) WithOption and renamed it just Option, but that's not a breaking change due to continuing to keep the old WithOption type in place.

So, I think (unless you say otherwise) this should be "safe" to cut as another 0.x release. :)

First and foremost. This is great work. Kudos!

Now: I tend to agree the happy/common path should be relatively safe/straightforward, or so it seems reviewing the changes.
I also agree the most affected packages (like pkg/options) are not expected to be used in isolation, so the blast radius is contained.

And, truth is, the more I review the more I like this change, so I would really like to have it in. But I'm still worried about how breaking this breaking change is. I think stability is a good plus, and while this change has a lot of value for maintainers, I don't see a compelling case for users of ghw. They will face some disruption, but what would they get?

If we can wither minimize the disruption (avoid it completely would be ideal ofc) AND clearly outline the value proposition, I would be much more comfortable and we can just go ahead.
I'm struggling to see how we can implement this cleanly while minimizing the disruption, though.

What I can do next is to try out a ghw snapshot with this PR applied against a couple of non trivial codebases and see how it looks like.
ATM, however, I'm inclined to NOT merge unless we also bump the X version (X.Y.Z)

Some minor comments inside; I didn't go much further because, well, the PR is already pretty clean and I'd rather settle out the compat. conversation before to dive into details.

[jaypipes]

@ffromani thank you, as always, for the thoughtful review and commentary. I very much appreciate your focus on stability and backwards compatibility!

There are actually only two backwards-incompatible changes to the API/ABI included in this PR, both of which I believe are only applicable to a very small slice of users (perhaps only you and I ;) ):

1. pkg/memory.CachesForNode(). This function's signature changed from accepting a *pkg/context.Context argument to accepting a *pkg/option.Options argument. This function is exported only because it's used by pkg/topology. Otherwise, it's an undocumented API call that really wasn't usable by anyone outside of the ghw devs anyway since you'd need to know how to manually construct a pkg/context.Context thing.
2. The Snapshot functionality has changed, quite obviously. The ghwc-snapshot tool has been removed and replaced with a -s flag on the main ghwc program. Additionally, the pkg/snapshot functions have been changed and consolidated. Again, though, the pkg/snapshot functions I believe were only ever used by ghwc--snapshot and ghwc-snapshot is only ever used by either us or by a bug reporter after being asked by us to run it :)

All other changes to the main pkg/XXX.New() functions are actually backwards compatible. If a user uplifts their go.mod import of ghw, their code will compile as-is because of the type-aliasing in alias.go.

For example, if the user's code looked like this:

package main

import "github.com/jaypipes/ghw"

func main() {
    cpu, err := ghw.CPU()
    if err != nil {
        ...
    }
}

nothing has changed about the call signatures for either the type-aliased ghw,CPU() or non-type-aliased pkg/cpu.New() functions, so the above code will compile without any changes needed by the user.

Same for the following:

package main

import "github.com/jaypipes/ghw"

func main() {
    cpu, err := ghw.CPU(ghw.WithDisableTools())
    if err != nil {
        ...
    }
}

or this:

package main

import (
    "github.com/jaypipes/ghw"
    "github.com/jaypipes/ghw/pkg/option"

func main() {
    cpu, err := ghw.CPU(option.WithChroot("/mnt/newroot/")
    if err != nil {
        ...
    }
}

or even this:

package main

import (
    "github.com/jaypipes/ghw/pkg/cpu"
    "github.com/jaypipes/ghw/pkg/option"

func main() {
    info, err := cpu.New(option.WithChroot("/mnt/newroot/")
    if err != nil {
        ...
    }
}

I'm fine creating a v1 release series git branch and redirecting this PR to that branch, but AFAICT, apart from the minor incompatible things related to pkg/memory.CachesForNode() and the snapshotting code, it's actually backwards-compatible... anyway, I'll leave it to you what you want to do. Happy to do the v1 branch thing but also happy to do a v0 dot release.

[ffromani]

Thanks @jaypipes!

Again, your points make sense and the code looks good. I'm not by any means trying to diminuish this work! I can't think in any way I could have done better. Thing is, I've seen ghw (or any sufficient widespread package) used in really creative ways, this is the reason why I'm most careful (and yes, perhaps I'm overly careful ;) ).

I guess I need to convince myself trying by doing the upgrade as I mentioned in the previous comment. I will attempt shortly. Thanks again for the patience and the explanation, and apologies for the delay in review.

[ffromani]

thinking about it, I have a proposal. What if we identify few (in the order of like 4-5) prominent importers of ghw and use them as benchmark when evaluating changes like this?

Something like "project XYZ compiles with no changes and passes all the tests" or "project ABC needs this patch but then compiles cleanly and passes all the tests" or (hopefully not) "project FOO compiles with no changes but fails tests 1, 4, 77".

EDIT: obviously I volunteer to help implementing this proposal, getting in touch with these projects, submitting test PRs and socializing the changes.
EDIT2: but I would need help creating (and maintaining) this shortlist. And I'm not saying this proposal, or the implementation thereof, should be necessarily gating for this PR even if we like it and implement it; that's a separate conversation and I have no real preference.

[jaypipes]

thinking about it, I have a proposal. What if we identify few (in the order of like 4-5) prominent importers of ghw and use them as benchmark when evaluating changes like this?

Something like "project XYZ compiles with no changes and passes all the tests" or "project ABC needs this patch but then compiles cleanly and passes all the tests" or (hopefully not) "project FOO compiles with no changes but fails tests 1, 4, 77".

EDIT: obviously I volunteer to help implementing this proposal, getting in touch with these projects, submitting test PRs and socializing the changes. EDIT2: but I would need help creating (and maintaining) this shortlist. And I'm not saying this proposal, or the implementation thereof, should be necessarily gating for this PR even if we like it and implement it; that's a separate conversation and I have no real preference.

I like the proposal. What would be your short-list of importing projects?

[ffromani]

thinking about it, I have a proposal. What if we identify few (in the order of like 4-5) prominent importers of ghw and use them as benchmark when evaluating changes like this?
Something like "project XYZ compiles with no changes and passes all the tests" or "project ABC needs this patch but then compiles cleanly and passes all the tests" or (hopefully not) "project FOO compiles with no changes but fails tests 1, 4, 77".
EDIT: obviously I volunteer to help implementing this proposal, getting in touch with these projects, submitting test PRs and socializing the changes. EDIT2: but I would need help creating (and maintaining) this shortlist. And I'm not saying this proposal, or the implementation thereof, should be necessarily gating for this PR even if we like it and implement it; that's a separate conversation and I have no real preference.

I like the proposal. What would be your short-list of importing projects?

Since my proposal is related to the breaking changes assessment, but is not necessarily gating, I'm moving the conversation here: #421 and I sketched an initial very rough draft of a shortlist and the criterias to add/keep projects in the list

Will try out #417 (comment) as soon as possible

[ffromani]

update: testing is going good! I need to run a few more tests (disclosure: going slower because vacation) but so far so good, looks like we're trending towards merge!