Use the WNFS Append Endpoint

[jessmartin]

1/ Who will benefit from this idea?

Discourse users who would like to save their files to IPFS.

2/ What is the problem or opportunity?

Discourse by default stores uploaded files locally. This is problematic in case the server goes down. Some people migrate their files to S3-compatible object storage. Those storage systems are expensive and require maintenance to secure and backup. Additionally, the files could end up locked in the s3 bucket if the application goes away due to not having a publicly-available URL. Files have to be routed through Discourse.

Writing files to IPFS would be cheaper, lower maintenance, as well as storing the files with a public URL to be referenced.

The plugin currently supports an outdated mechanism for writing to IPFS via a password-protected endpoint. If we swap out this mechanism for the new Append endpoint that will soon be added, we can get the plugin working again.

3/ What is the most important benefit to people?

Honestly, I'm not sure which of the benefits are most important (public URL, cheaper, no maintenance necessary).

4/ How do you know what people want?

Not exactly sure. Not sure if this plugin has or ever had any users. Fission will be an initial user for Fission's Talk Discourse instance.

5/ What does the experience look like?

Admin setup will be very similar to the current plugin implementation:

1. Use the Fission CLI locally to create a key.
2. Visit $your_site/admin/site_settings/category/plugins?filter=ipfs_storage
3. Check Enable IPFS File Storage
4. Paste the contents of the root key into Settings.

Once the Discourse instance is configured to write to IPFS via the Append endpoint, the user experience of uploading files to a post will be unchanged. Files will be persisted to IPFS and the URL will be able to be referenced publicly.

[jessmartin]

@bmann Would love your eyes on this to make sure I'm capturing the problem appropriately. Also, should this have been created in Talk or here? I created it here so that it could be added to the Everything Board.

@bgins I know we talked about UCAN creation today, but it's not clear to me how a Discourse instance would go about creating credentials. Any examples of this I can reference? Maybe there's an "account configuration" screen in Discourse?

[bgins]

One way I can imagine this working is an integration in the user Preferences > Account page. There are integrations for other services there:

Maybe there could be a Fission one there?

In terms of getting the credentials there, I think the following would work:

• User clicks on Fission "Connect" button
• Redirect to the Fission Auth Lobby with a request for app and fs permissions
• Redirect back to Discourse
• Auth lobby sends UCAN over window post message to Discourse
• Discourse stores the UCAN
• Use the UCAN when calling the append endpoint

The fs permissions would be restricted to the public directory where the append endpoint will store data:

public/apps/<creator>/<app-name>/uploads/

With the exception of using the UCAN, I think this could all be done with webnative today:

• Call webnative.initialize with app and fs permissions and loadFilesystem set to false
• Call redirectToLobby with the permissions

The creator and app-name are determined by the app permissions provided to webnative.initialize and redirectToLobby.

I think that setting loadFilesystem to false should be enough to prevent Discourse from running all of WNFS. It might be worth experimenting with this in a small POC app, would be happy to help out with that some time if would like.

[jessmartin]

@bgins Thanks for the response!

One way I can imagine this working is an integration in the user Preferences > Account page.

Just a reminder that this is a "per-instance" setting, not a "per-user" setting.

After wading around in the Settings for a while (both Admin and User) and comparing to other plugins, I think the right way to do this is in the Admin settings section. The current plugin adds a few options under the Admin section. See below:

I think we'll want to do something similar. It's a little strange in that the flow will technically be setting up a Fission account for a user, not an app, but I think it's the right way to do the configuration because all Admins will have access to the settings, rather than them being nested under the Personal settings section.

---

From @bmann: "basically, since we don't have "per app UCAN", you would use the key itself, from your CLI"

Boris suggests we just use the CLI to generate a key locally. That would be... doing it from the command line and storing the key somewhere in the filesystem that Ruby can read? Clunky, but I believe it would work.

---

I think @bgins' flow for "Connect to Fission" from the Admin settings will still work, though! I'll see if I can get a little POC plugin set up that goes the process. A few notes and thoughts below.

Reminder that the code to save to send the file to the Append endpoint is in the Ruby backend.

• Discourse stores the UCAN

I take that to mean: the Discourse admin frontend will use an internal API to write the UCAN to the Discourse db (or filesystem?) to retrieve for later usage. Yesterday we talked about how the private key won't be exportable. Will the public key / DID be sufficient to "sign" files posted to the Append endpoint?

Also, in this case, that private key will be forever locked into the browser whichever Admin happens to set up the setting with. That... doesn't seem good?

I think that setting loadFilesystem to false should be enough to prevent Discourse from running all of WNFS.

I'm not sure what "running all of WNFS" means?

[jessmartin]

@bgins Ok, had a quick discussion with @bmann in Discord that clarified/simplified things. I'm updating the description of this Issue to reflect my new understanding.

[bgins]

Just a reminder that this is a "per-instance" setting, not a "per-user" setting.

Ah right, per instance, not per user. Makes sense. 👌

I was confused about where we will be calling the append endpoint. If we are calling it from the server, that changes things.

Boris suggests we just use the CLI to generate a key locally. That would be... doing it from the command line and storing the key somewhere in the filesystem that Ruby can read? Clunky, but I believe it would work.

Yup, a bit clunky but seems alright to me to get it working.

I take that to mean: the Discourse admin frontend will use an internal API to write the UCAN to the Discourse db (or filesystem?) to retrieve for later usage. Yesterday we talked about how the private key won't be exportable. Will the public key / DID be sufficient to "sign" files posted to the Append endpoint?

I meant that Discourse would store the UCAN in the browser in IndexedDB, but this doesn't make sense because we are saving files from the Ruby backend. Having the key locally on the server seems simpler and a better way to go.

[bmann]

Yes. Just like GH Actions publish. We store a full DID key corresponding to the “app” for the Discourse server.

this is “web2 compat” mode. Where we make it as easy as possible for web2 apps to publish to IPFS

Most of the world doesn’t speak UCAN today, so we fall back to UCAN as API key. This is upgradable over time and we want to generate UCANs per service / purpose — just like you do with OAuth keys.