Commit Graph

133 Commits

Author SHA1 Message Date
shockrah
cfbee6d887 ! This merge denotes the MVP completion for the backend decentralization endpoints
More routes are to be added in the future but for now this should suffice

Doc's are now required and the roadmap need to be updated as well
2021-05-12 14:27:34 -07:00
shockrah
c25e8047cf + Finishing up more tests for the /neighbors routes
For now this sub api is at an mvp stage so it should be fine to move onto new
features after this point for now.
2021-05-12 14:22:14 -07:00
shockrah
24b5f11b13 * Better logging in meta module(more logging tags) 2021-05-12 14:14:30 -07:00
shockrah
fbcf16b735 + Adding /neighbor/remove route api code and db-lib code
This patch has not yet been tested but will be in the coming patches

Sumamary: we're just doing a regular delete on the neighbor's table based off
the url field
2021-05-12 14:07:38 -07:00
shockrah
e94c720332 + More comprehensive testing on /neighbor/update
The cases that actually matter should be covered now so I'm confident with this
endpoint's behavior
! A preliminary db::neighbors::get is done to cover the 404 case
Basically if a bad url is used in the request we should 404 the update as there
won't be anything relevant to update
2021-05-12 13:05:52 -07:00
shockrah
8bff61ab71 * Fixed issue where put requests weren't firing with the optional body parameter
This also "fixes" the /neighbor/update route conveniently enough, which had much
better behavior than expected before.

- Remvoing some fluff from debugging
2021-05-12 13:02:42 -07:00
shockrah
a0d60c0568 - Pointless permissions check /message/send handler
Auth module literally does this for us see: auth::valid_perms
2021-05-11 17:27:04 -07:00
shockrah
a941165ea5 + adding /neighbor/update route to dispatch
! This route needs way more testing as its currently failing the prelim test listed right now

! Some more misc changes in testing/mod.rs that aren't imporant at all to anyone
It's just an extra comment
2021-05-11 17:25:51 -07:00
shockrah
1ca17ec6e0 * /neighbor/add handler now uses body for neighbor structure data
This chage is basically required due to the ridiculuous number of parameters that needed to be passed.
Also more POC for showing how serde_json's result type can be used to safely parse http bodies
2021-05-11 17:23:15 -07:00
shockrah
666894af0e + Debug to Neighbor struct
* add_neighbor now takes a moved Neighbor param instead of the fields individually
This should make it a easier for both front & backend to parse what's happening.
It also let's us leveraege serde_json a bit more
2021-05-11 17:18:17 -07:00
shockrah
664b837f13 + More tests for /neighbor routes
! put /neighbor/update is failing due to a failure to parse the body correctly on thebackend
Further investigation is required

+ Kinda minor but I'm also adding the ability to `put` things now
+ Also an if statement to avoid failing on checkf for NoneType responses
2021-05-11 17:16:50 -07:00
shockrah
b4c55b72ea - Removing unused .env file
diesel still requires this however the --database-url flag is there to alleviate this issue
+ make test now cleans up fake png images from the cwd
This is more for comfort reall
2021-05-11 17:14:22 -07:00
shockrah
45540ddd25 * Better failure logging in failed requests
self.passing property is a much more comprehensive way of checking for passing tests
Also this uses the colors to dump to stdout
2021-05-11 15:22:56 -07:00
shockrah
b810a5abba * Fixing message body unwrap to safer unwrap_or
unwrap had a chance to panic where as the unwrap_or now defaults to an empty
Bytes object which lets the code after it fail with an HTTP 400. This is preferrable
to a 500 error as there really is no error, just bad/unpollable input from the
end user.
2021-05-11 13:52:07 -07:00
shockrah
f1b1581588 + db::neighbors::get and db::neighbors::update
It should be noted at this point we're considering url's as the "primary key"
even though its kinda weird to use them that way.
Even though a varchar primary key doesn't really scale well I'm considering it
fine for now as the neighbors table is controlled by admins(by hand) and hopefully
won't need to scale.
2021-05-11 13:45:41 -07:00
shockrah
83f846640c - Removing unused .env files
Opting now for config.json files as they better interoperability between node and rust

+ Hardcoding some variables into build-db.sh
This really shouldn't be an issue however

+ using cat to dump out what table structures are being setup due to strange bugs in pipeline
2021-05-10 15:58:57 -07:00
shockrah
96bdd70cd5 + MOre comprehensive logs from subprocess in create_admin 2021-05-10 13:59:10 -07:00
shockrah
ee5d9fb248 + Better formatting in hyper compact code
+ Adding ADD_NEIGHBOR permissions flag
2021-05-09 23:16:43 -07:00
shockrah
b3c27b86ce * Fixing some json serialization/parsing weirdness in the meta::BASIC_CONFIG initialization
This is part of a bigger change to stop using environment variables around as state as its mega cheese
2021-05-09 23:15:09 -07:00
shockrah
adc5b261e8 + ADD_NEIGHBOR route now supported in API backend
! Requiring a special event in the RTC server docs for this change
2021-05-09 23:14:02 -07:00
shockrah
c443b9bb07 + More tests for the /neighbor/add and /neighbor/list routes
These are baseline tests however a new /neighbor/remove & /neighbor/edit route should be edited before I call this done on the roadmap.
Also some more intense testing around these current routes is required.
Mostly because the expectation that JSON is being sent  to us in /neighbor/add
It could be worth the effort to send this data as json in the body
! Currently parameters are sent via the query string is in line with how most routes behave
For this route is just feels weird dealing with al the issues with json in the query string
2021-05-09 23:11:51 -07:00
shockrah
a79195076d * Changin public_url & public_ws_url to shorter url & wsurl respectively
There's also some miscellaneous changes  that don't fit anywhere in this patch (mostly dev qol of roadmap updates)
2021-05-09 23:08:55 -07:00
shockrah
e8348918c4 + More qol like better __str__ methods everywhere
+ More error handling in case shit goes wrong
Basically handling more cases where some initial test could result in the whole
script exiting with code(1)
Not really that big of a deal since most tests after _that_ point will fail anyway
but the fix has revealed issues in the auth code magically so I'm keeping up with the new idea
that initial tests should have every resultant case validated to avoid weird behavior

> good code results in good results
who would have guessed
2021-05-08 02:07:32 -07:00
shockrah
cdb956a85c Minor changes lumped together
* claim.rng is now seeded from 16 bytes
+ passing Member to auth::login_get_jwt
* Better error loggin on jwt::insert call

+ Moar meta config-caching fixes
Basically lazy static sucks and we have to start accessing meta::basic_config
though a proxy function which forces the initialization on startup
! We should probably init this prior to listening for connections to avoid connection issues

+ New /neighbors/list endpoint which has already passed required checks
2021-05-08 02:03:58 -07:00
shockrah
1c366611d9 + Server tags are now stored in json format(still in the environment variable mind you
Lazy static initialization still allocates once per run so the above is jank but fine(mostly)
	[] Should probably come up with a cleaner way of passing this data around
	The above is very low on the priority list of things to do

+ Now injecting permisions + id via the claim structure which is passed to the
route dispatcher
! For now this is unused further down the decision tree however I'm going to sprinkle
in its incorporation as it lets us avoid string conversions which is p nice
2021-05-08 01:59:31 -07:00
shockrah
c850d42ce1 + Jwt tables - SEE NOTE
! wat - because have to do maintain permissions on a per request level we have
to do this check for permissions at what is basically every level, this does
mean we have to hit the database for a lot of routes however there is a check
that requests go through in order to avoid hitting the database whenever possible

+ rng field in claims now has real purpose
It's purpose is to act as a validator field in the jwt table. By verifying rng
fields we no  longer have to store whole jwt's
2021-05-08 01:29:44 -07:00
shockrah
9a22713080 + Testing /invite/join
+ Testing /meta
+ Testing /neighbors/list
2021-05-06 17:12:49 -07:00
shockrah
34115477ab + Mostly adding quality of life changes in this patch with the mock client
These changes will make it easier to read through verbose logs with some tactful
colors (yellow)
2021-05-06 17:03:24 -07:00
shockrah
181b1cadc4 + Exposing serde_json to db-lib lyaer
This dep was already there but just being used (as we're using serde). Now we can use it at the db layer which makes neighbor struct creation easier
* Cargo locks are being updated to reflect new dependancy changes
These changes actually reduce the dependancy count overall
+ Adding a very simple table to hold all the neighbors that an instance may want to reference
At some point we may want to support vanity join urls and this would be a good place to reference those
There would also need to be a way for admins to add/edit/remove vanity urls but that's for another patch
2021-05-06 13:29:32 -07:00
shockrah
a628d56e25 - Removing dotenv from required dependancies
+ Leveraging serde_json to use a config.json
This lets us use the same config for both the json-api and the rtc-server without adding dependancies
2021-05-05 15:50:39 -07:00
shockrah
f094ab49de * Adding some safey into the mock-client tester so that it doesn't just explode in pipelines 2021-05-03 13:24:24 -07:00
shockrah
7c8bc8b4fc - Removing warning
! Increasing wait time on client mock script to avoid firing too early
2021-05-01 19:16:18 -07:00
shockrah
34426038d9 * Fixing busted invite test in api client tester
+ Adding some more invite status checks
+ Adding a verbose invite usage check for sanity reasons

- Removing run-api-tests script as it now goes into the /scripts/ directory

+ Adding the wss-hmac setup from the command line arg
It was literally just 1 if statement that I forgot to write in
2021-04-27 13:27:39 -07:00
shockrah
61a3f1d4f5 + Adding first pass of a pair of invite endpoint tests 2021-04-26 02:01:04 -07:00
shockrah
c9658ad5b4 + Adding command line flag [-H/--hmac] to specify hmac file path
This is really just for testing purposes since the files tend to be in
awkward to reach path

- Removing if args.len == 0 check
Basically cannon-fodder tbh

* Auth now reads hmac path from environment var

! All of the above is added for the wss hmac as well

+ Adding command line flag [-W/--wss-hmac] to specify wss-hmac file path
2021-04-26 01:58:18 -07:00
shockrah
90b584eaa2 * Fixing /invites/join as it was returning the wrong struct data 🤦
Secret value given to user was literally the encrypted secret because I forgot
to rebind it

- removing dev key lmaoooo
These are generated super easily and this key doesn't actually go to anything
so its really not a problem, although it is annoying that its there
2021-04-25 12:40:08 -07:00
shockrah
e2e68c51ba - Removing old client test code in favor of new structure
+ Moving tests to the new client
- Removing web/ module
! Currently all tests are passing 17/17 but the real trickery comes with doing this
on CI which should will likely take some magic somewhere
Or we'll just extend the freechat docker image to finally have all the required
dependancies and just test on that with diesel and what not
2021-04-21 21:38:25 -07:00
shockrah
9ab9cdb176 + db-lib::Channel::get added and Returns a Result<Option<Channel>> for simplicity
This is really only for check if a particular channel exists for /messages/recent's backend

* Flattening db-lib::Message::last_n
SHould make code more readable as it now has a more linear flow

* api::Messages::recent_messages now 404's when a channel that doesn't exist is request
2021-04-21 21:33:30 -07:00
shockrah
432e94054f Fixing minor errors that were preventing tests fro mbeing ran oops :p 2021-04-21 17:23:00 -07:00
shockrah
38ff0edd39 ! Massive test suite overhaul see details below !
Problem: the old test suite was extremely inflexible
This meant that making new tests took way too much time.

+ This new rework makes the new client's backend much thinner and less "magical"
With less magic going on we can pass way more data more easily to the actual
http-request engine making the convenience wrapper over top it much more flexible

Translating old tests to the new engine might take a while but for now the old
client codebase is completely deprecated and will no longer be used+updated
2021-04-21 17:22:16 -07:00
shockrah
c4d7eb9111 * api::channels::list_channels has more relevant docs
Better matching against return types so we're not throwing away errors
	This just entails swapping the if let for a match statement

* db-lib::Message::send has more updated docs on what variants of Response it returns

* api::messages::send_message has better error logging now and more relevant comments
2021-04-21 17:14:33 -07:00
shockrah
ef73b39f4f + Adding db-lib::common::try_error_passthrough
This is more heavily used in Message::send as a way of discerning "real" server errors
from those that are caused  by user input. The name itself might not be super fitting either

* Moving code for inserting textual messages into its own function
This splits up the logic a bit for Message::send but this segment of logic is also
much simpler than that of file upload

* Flattening Message::send overall
Keeping this function as flat as can be is crucial as it is one of the heavier+most
important funcs in the whole JSON API codebase

Files are now also generated correctly and asynchronously
2021-04-21 16:08:23 -07:00
shockrah
52676cdd1f - Removing /messages/from_id as its no longer usable with the new id generation model
+ Adding new flags to sample .env file
+ Wrapper script run-api-tests.sh which does as its called
- Removing build.sh in favor of run-api-tests.sh
Makefile takes care of building, and apitests are ran with special script anyway
2021-04-20 12:09:05 -07:00
shockrah
bb2201c00e ! First pass base implementation for /messages/recent
This api method is made for clients to easily say "give me the n[1,100] latest
messages in some channel
This commit contains code that is largely untested but contains some base code
such that I can correct issues in the next _tested_ commit
2021-04-17 15:55:10 -07:00
shockrah
522890fa00 + rtc::new_message now takes a Pool ref to make the second call for the user name
! Start taking public user params like this in the JWT to reduce on network hits
It's cheap and reliable enough but the idea just now came to me so do that at some point
2021-04-14 22:46:31 -07:00
shockrah
89bd257213 * More explicit query string guard in /channels/list endpoint handler
* rtc::make_url now builds websocket urls more robustly incase we have to percent
encode them
2021-04-10 16:45:13 -07:00
shockrah
283c201e96 * More explicit query string guard in /channels/list endpoint handler
* rtc::make_url now builds websocket urls more robustly incase we have to percent
encode them
2021-04-10 16:44:24 -07:00
shockrah
368c9bc95f + Better logging in main
+
2021-04-07 21:06:08 -07:00
shockrah
d030accddc * Adding id to insertion call in db-lib::channel::add
This is done at the driver level because the member id's are randomly generated
u64's
2021-04-07 20:48:58 -07:00
shockrah
3f1dfbf824 Adding rtc notifications to relevant endpoints
! Still need a feature flag to compile this optionally instead of all the time
- Removed warning in main's auth::login_get_jwt call
2021-04-02 13:56:44 -07:00