This patch has not yet been tested but will be in the coming patches
Sumamary: we're just doing a regular delete on the neighbor's table based off
the url field
The cases that actually matter should be covered now so I'm confident with this
endpoint's behavior
! A preliminary db::neighbors::get is done to cover the 404 case
Basically if a bad url is used in the request we should 404 the update as there
won't be anything relevant to update
This also "fixes" the /neighbor/update route conveniently enough, which had much
better behavior than expected before.
- Remvoing some fluff from debugging
! This route needs way more testing as its currently failing the prelim test listed right now
! Some more misc changes in testing/mod.rs that aren't imporant at all to anyone
It's just an extra comment
This chage is basically required due to the ridiculuous number of parameters that needed to be passed.
Also more POC for showing how serde_json's result type can be used to safely parse http bodies
* add_neighbor now takes a moved Neighbor param instead of the fields individually
This should make it a easier for both front & backend to parse what's happening.
It also let's us leveraege serde_json a bit more
! put /neighbor/update is failing due to a failure to parse the body correctly on thebackend
Further investigation is required
+ Kinda minor but I'm also adding the ability to `put` things now
+ Also an if statement to avoid failing on checkf for NoneType responses
diesel still requires this however the --database-url flag is there to alleviate this issue
+ make test now cleans up fake png images from the cwd
This is more for comfort reall
unwrap had a chance to panic where as the unwrap_or now defaults to an empty
Bytes object which lets the code after it fail with an HTTP 400. This is preferrable
to a 500 error as there really is no error, just bad/unpollable input from the
end user.
It should be noted at this point we're considering url's as the "primary key"
even though its kinda weird to use them that way.
Even though a varchar primary key doesn't really scale well I'm considering it
fine for now as the neighbors table is controlled by admins(by hand) and hopefully
won't need to scale.
Opting now for config.json files as they better interoperability between node and rust
+ Hardcoding some variables into build-db.sh
This really shouldn't be an issue however
+ using cat to dump out what table structures are being setup due to strange bugs in pipeline
These are baseline tests however a new /neighbor/remove & /neighbor/edit route should be edited before I call this done on the roadmap.
Also some more intense testing around these current routes is required.
Mostly because the expectation that JSON is being sent to us in /neighbor/add
It could be worth the effort to send this data as json in the body
! Currently parameters are sent via the query string is in line with how most routes behave
For this route is just feels weird dealing with al the issues with json in the query string
* Hopefully fixing pipeline script
Cargo kept trying to rebuild everything so hopefully the new change
forces a usage of artifacts that _should_ be there
+ More error handling in case shit goes wrong
Basically handling more cases where some initial test could result in the whole
script exiting with code(1)
Not really that big of a deal since most tests after _that_ point will fail anyway
but the fix has revealed issues in the auth code magically so I'm keeping up with the new idea
that initial tests should have every resultant case validated to avoid weird behavior
> good code results in good results
who would have guessed
* claim.rng is now seeded from 16 bytes
+ passing Member to auth::login_get_jwt
* Better error loggin on jwt::insert call
+ Moar meta config-caching fixes
Basically lazy static sucks and we have to start accessing meta::basic_config
though a proxy function which forces the initialization on startup
! We should probably init this prior to listening for connections to avoid connection issues
+ New /neighbors/list endpoint which has already passed required checks
Lazy static initialization still allocates once per run so the above is jank but fine(mostly)
[] Should probably come up with a cleaner way of passing this data around
The above is very low on the priority list of things to do
+ Now injecting permisions + id via the claim structure which is passed to the
route dispatcher
! For now this is unused further down the decision tree however I'm going to sprinkle
in its incorporation as it lets us avoid string conversions which is p nice
! wat - because have to do maintain permissions on a per request level we have
to do this check for permissions at what is basically every level, this does
mean we have to hit the database for a lot of routes however there is a check
that requests go through in order to avoid hitting the database whenever possible
+ rng field in claims now has real purpose
It's purpose is to act as a validator field in the jwt table. By verifying rng
fields we no longer have to store whole jwt's
This dep was already there but just being used (as we're using serde). Now we can use it at the db layer which makes neighbor struct creation easier
* Cargo locks are being updated to reflect new dependancy changes
These changes actually reduce the dependancy count overall
+ Adding a very simple table to hold all the neighbors that an instance may want to reference
At some point we may want to support vanity join urls and this would be a good place to reference those
There would also need to be a way for admins to add/edit/remove vanity urls but that's for another patch
Was basically pointless since it's literally just two commands
! full-mock-test job now has scripts/run-api-tests.sh as its source code
This makes it way easier to debug the pipeline when things go wrong as there's
lots of _different_ things going on in that script.
While the script itself is still there it is completely deprecated and safe to remove
at this point
! PAIN Peko
Dependancies might be fucked because gitlab-runner doesn't use deps at all and frankly
I'm completely over doing this stupid patch. Next is decentralized features which is way
more fun to do so if it breaks I'll fix it but its likely an issue with artifacts
* Removing export command from cargo-unit-test job
This was causing the job to fail so the env vars have been moved to live under
the ci file's job-level variables key
* Also I've updated the docker image that this pipeline uses as it didn't have node
and pip installed for some reason
Link to relevant docker image: https://hub.docker.com/r/shockrah/fc-pipeline
* Increasing expries_in flags to be 30 minutes from 15
+ Adding build-keys job which builds hmac keys
+ build-rtc-server: simple build job
* Moving cargo-test-json-api (unit tests) to the bottom with the other tests
! full-mock-tests are now implemented so its time to watch them fail horribly
+ Adding new bootstrapping script for Docker testing pipeline
This build-database script is what is going to setup things like our tables for us
so that we can actually run our test queries
! Basically turning off the ci right now since its going through some large breaking changes
It's going to be broken for a while so there's no point in running it into
a wall of predictable failures
+ Adding some more invite status checks
+ Adding a verbose invite usage check for sanity reasons
- Removing run-api-tests script as it now goes into the /scripts/ directory
+ Adding the wss-hmac setup from the command line arg
It was literally just 1 if statement that I forgot to write in
This is really just for testing purposes since the files tend to be in
awkward to reach path
- Removing if args.len == 0 check
Basically cannon-fodder tbh
* Auth now reads hmac path from environment var
! All of the above is added for the wss hmac as well
+ Adding command line flag [-W/--wss-hmac] to specify wss-hmac file path
Secret value given to user was literally the encrypted secret because I forgot
to rebind it
- removing dev key lmaoooo
These are generated super easily and this key doesn't actually go to anything
so its really not a problem, although it is annoying that its there
+ Moving tests to the new client
- Removing web/ module
! Currently all tests are passing 17/17 but the real trickery comes with doing this
on CI which should will likely take some magic somewhere
Or we'll just extend the freechat docker image to finally have all the required
dependancies and just test on that with diesel and what not
This is really only for check if a particular channel exists for /messages/recent's backend
* Flattening db-lib::Message::last_n
SHould make code more readable as it now has a more linear flow
* api::Messages::recent_messages now 404's when a channel that doesn't exist is request
Problem: the old test suite was extremely inflexible
This meant that making new tests took way too much time.
+ This new rework makes the new client's backend much thinner and less "magical"
With less magic going on we can pass way more data more easily to the actual
http-request engine making the convenience wrapper over top it much more flexible
Translating old tests to the new engine might take a while but for now the old
client codebase is completely deprecated and will no longer be used+updated
Better matching against return types so we're not throwing away errors
This just entails swapping the if let for a match statement
* db-lib::Message::send has more updated docs on what variants of Response it returns
* api::messages::send_message has better error logging now and more relevant comments
This is more heavily used in Message::send as a way of discerning "real" server errors
from those that are caused by user input. The name itself might not be super fitting either
* Moving code for inserting textual messages into its own function
This splits up the logic a bit for Message::send but this segment of logic is also
much simpler than that of file upload
* Flattening Message::send overall
Keeping this function as flat as can be is crucial as it is one of the heavier+most
important funcs in the whole JSON API codebase
Files are now also generated correctly and asynchronously
+ Adding new flags to sample .env file
+ Wrapper script run-api-tests.sh which does as its called
- Removing build.sh in favor of run-api-tests.sh
Makefile takes care of building, and apitests are ran with special script anyway
This api method is made for clients to easily say "give me the n[1,100] latest
messages in some channel
This commit contains code that is largely untested but contains some base code
such that I can correct issues in the next _tested_ commit
+ Better error logging in cache
! There is still lots of debug logs which at some point need to be built _out_ from prod builds
Also I need some prod builds
+ Adding more return type annotations for clarity/docs sake
- Removed ancient docs in messages module as they were basically just straight up wrong
+ messages module was for some reason still using legacy url generation(fixed that)
+ switching types to use bigint since everything is u64/i64 on the backend
Interface for this is fiddly and probably requires real docs to be further developed
without losing my mind doc the whole cache at some point
+ New cache ipc handlers for adding a new open web socket
Web socket comes with some basic listeners, however very litte/nothing
is being done check the health of these connections or to try when possible.
+ Cache now adds actual message objects to its message arrays instead of raw strings (wew)
+ Events module has been added to move the parsing logic/validation away from everything else
+ The basic Event structure has some niftier-than-you-think behavior for data acccess which the cache can leverage for more concise+ correct behavior
! Start taking public user params like this in the JWT to reduce on network hits
It's cheap and reliable enough but the idea just now came to me so do that at some point
+ Cache is no longer contained in app structure
This should let us later throw this into an Arc<Mutex<>> for better
async support.
! Also more cache access safety is being done on the front end because the cahe doesn't guarantee much on access
Perhaps some convenience wrappers would make this look nicer(with inline)
!!! Main needs a lot of flattening
It's technically not much of a problem since most of the code behind 1/2 match's
which really just bloat the hell out of indentation making it look bad when its not _that_ bad.
However it still bugs me so I should probably do something about that(also (inline))
From this point forward the cache should basically just be a data container.
Methods on the cache object are there for convenience and should only ever
return very simple data [booleans, integers, () etc.].
The cache should also have very litter/no concept of commands to decouple it
from the renderer process.
! list_channels is still implemented in the cache however it remaains a vestige
and will be moved to the net module
Also the client is getting some new packages for websocket connections and
while they work some additional work has to be put in to figure out where fug to
actually place the websocket so that it can update the cache+DOM in a seomewhat sensible fashion
Json-bigint should help alleviate this issue by correctly parsing numbers for us
- Removing url method in favor of new url property in server config model
+ Building UserConfigs and ServerConfig from typical JS Obects
This should reduce some up front complexity regarding parameters
! Fix: using new url property instead of protocol + host + port
- Reducing upfront complexity of public channels API
* Also moving to using new updated server.url property
* Channels also handles its part of updating the dom now
bigint parsing in configs now
* More switches to server.url property
Of course this is an issue because we can no longer get data from the user
A full rearchitecting may be in order here however its not impossible to modify the current architecture so far
The main issue here is that the cache is doing too much I think
If the termion(main input) task does its thing, and a socket task,
waits around waiting to build a sub task(socket) then we may be able to
pass messages to the socket task to open up sockets when request
This allows us to have shared state via message passing and a main task
can take care of updating the cache.
The renderer can then request data from the cache when it needs to render stuff
* Jwt is now given over the query string as many websocket libraries make
it hard to pass header parameters
Parsing these values should also work fine but we'll see in time
Issue was that we weren't sending the correct headers/body data(ya I know)
- Removing mouse control from app since its completel irrelevant
!+ Putting app messagses behind a mutex for now since they are written to by
sockets and renderer process in the next few patches
More agressive channel caching may be required here but it works as an MVP for now
* Switch to hashmap for container of channel caches instead of vector
This should help with cache accesses as we don't have to iterate anymore
Also it makes data fetches much cleaner/shorthanded
* net::list_channels has its own private structure for collecting channels vector
This is basically required and is in the scope of the func itsefl to not pollute any other namespaces
! No wrapper goes around this module to avoid fragmentation
Note that basically every network hit is built around try's so they'll all return
HttpResult<T>'s
Technically everything is there but I haven't really tested anything yet
Will 100% need fixing + optimizing
+ Switch server Command also working
! Requires a jwt check to avoid expensive+unnecssary network /login hits
! Probably have to hold onto the input to avoid spam somehow
ChannelCache is nearly ready for manipulation
shockrah