Commit Graph
100 Commits
Author SHA1 Message Date
shockrah 34426038d9 * Fixing busted invite test in api client tester
+ Adding some more invite status checks
+ Adding a verbose invite usage check for sanity reasons

- Removing run-api-tests script as it now goes into the /scripts/ directory

+ Adding the wss-hmac setup from the command line arg
It was literally just 1 if statement that I forgot to write in
2021-04-27 13:27:39 -07:00
shockrah 2a1881db20 + Adding commandline parameter parser for specifying wss-hmac and api-hmac
The respective flags are -w|--wss-hmac and -H|--hmac
Default values are wss-hmac.secre hmac.secrett respectively
2021-04-27 13:24:59 -07:00
shockrah 61a3f1d4f5 + Adding first pass of a pair of invite endpoint tests 2021-04-26 02:01:04 -07:00
shockrah c9658ad5b4 + Adding command line flag [-H/--hmac] to specify hmac file path
This is really just for testing purposes since the files tend to be in
awkward to reach path

- Removing if args.len == 0 check
Basically cannon-fodder tbh

* Auth now reads hmac path from environment var

! All of the above is added for the wss hmac as well

+ Adding command line flag [-W/--wss-hmac] to specify wss-hmac file path
2021-04-26 01:58:18 -07:00
shockrah 1ee2f1a321 + Adding argparse to dependancy list 2021-04-26 01:34:29 -07:00
shockrah 225a874f5d Ignoring secret hmac files 2021-04-25 12:40:53 -07:00
shockrah 90b584eaa2 * Fixing /invites/join as it was returning the wrong struct data 🤦
Secret value given to user was literally the encrypted secret because I forgot
to rebind it

- removing dev key lmaoooo
These are generated super easily and this key doesn't actually go to anything
so its really not a problem, although it is annoying that its there
2021-04-25 12:40:08 -07:00
shockrah e2e68c51ba - Removing old client test code in favor of new structure
+ Moving tests to the new client
- Removing web/ module
! Currently all tests are passing 17/17 but the real trickery comes with doing this
on CI which should will likely take some magic somewhere
Or we'll just extend the freechat docker image to finally have all the required
dependancies and just test on that with diesel and what not
2021-04-21 21:38:25 -07:00
shockrah 9ab9cdb176 + db-lib::Channel::get added and Returns a Result<Option<Channel>> for simplicity
This is really only for check if a particular channel exists for /messages/recent's backend

* Flattening db-lib::Message::last_n
SHould make code more readable as it now has a more linear flow

* api::Messages::recent_messages now 404's when a channel that doesn't exist is request
2021-04-21 21:33:30 -07:00
shockrah 432e94054f Fixing minor errors that were preventing tests fro mbeing ran oops :p 2021-04-21 17:23:00 -07:00
shockrah 38ff0edd39 ! Massive test suite overhaul see details below !
Problem: the old test suite was extremely inflexible
This meant that making new tests took way too much time.

+ This new rework makes the new client's backend much thinner and less "magical"
With less magic going on we can pass way more data more easily to the actual
http-request engine making the convenience wrapper over top it much more flexible

Translating old tests to the new engine might take a while but for now the old
client codebase is completely deprecated and will no longer be used+updated
2021-04-21 17:22:16 -07:00
shockrah c4d7eb9111 * api::channels::list_channels has more relevant docs
Better matching against return types so we're not throwing away errors
	This just entails swapping the if let for a match statement

* db-lib::Message::send has more updated docs on what variants of Response it returns

* api::messages::send_message has better error logging now and more relevant comments
2021-04-21 17:14:33 -07:00
shockrah ef73b39f4f + Adding db-lib::common::try_error_passthrough
This is more heavily used in Message::send as a way of discerning "real" server errors
from those that are caused  by user input. The name itself might not be super fitting either

* Moving code for inserting textual messages into its own function
This splits up the logic a bit for Message::send but this segment of logic is also
much simpler than that of file upload

* Flattening Message::send overall
Keeping this function as flat as can be is crucial as it is one of the heavier+most
important funcs in the whole JSON API codebase

Files are now also generated correctly and asynchronously
2021-04-21 16:08:23 -07:00
shockrah 52676cdd1f - Removing /messages/from_id as its no longer usable with the new id generation model
+ Adding new flags to sample .env file
+ Wrapper script run-api-tests.sh which does as its called
- Removing build.sh in favor of run-api-tests.sh
Makefile takes care of building, and apitests are ran with special script anyway
2021-04-20 12:09:05 -07:00
shockrah bb2201c00e ! First pass base implementation for /messages/recent
This api method is made for clients to easily say "give me the n[1,100] latest
messages in some channel
This commit contains code that is largely untested but contains some base code
such that I can correct issues in the next _tested_ commit
2021-04-17 15:55:10 -07:00
shockrah 591e993329 + Adding cache handler for adding lots of messages from one server call
+ Better error logging in cache

! There is still lots of debug logs which at some point need to be built _out_ from prod builds
Also I need some prod builds

+ Adding more return type annotations for clarity/docs sake

- Removed ancient docs in messages module as they were basically just straight up wrong
2021-04-15 19:57:13 -07:00
shockrah 68e22e1b38 - removing uninportant things
+ messages module was for some reason still using legacy url generation(fixed that)
+ switching types to use bigint since everything is u64/i64 on the backend
2021-04-14 23:00:39 -07:00
shockrah 40351f934e + New cache ipc handlers for adding server caches
Interface for this is fiddly and probably requires real docs to be further developed
without losing my mind doc the whole cache at some point
+ New cache ipc handlers for adding a new open web socket
Web socket comes with some basic listeners, however very litte/nothing
is being done check the health of these connections or to try when possible.

+ Cache now adds actual message objects to its message arrays instead of raw strings (wew)
+ Events module has been added to move the parsing logic/validation away from everything else

+ The basic Event structure has some niftier-than-you-think behavior for data acccess which the cache can leverage for more concise+ correct behavior
2021-04-14 23:00:26 -07:00
shockrah 01c245cbba Ignoring more compiled js and building new events module 2021-04-14 22:48:41 -07:00
shockrah 522890fa00 + rtc::new_message now takes a Pool ref to make the second call for the user name
! Start taking public user params like this in the JWT to reduce on network hits
It's cheap and reliable enough but the idea just now came to me so do that at some point
2021-04-14 22:46:31 -07:00
shockrah 894a5bae34 - Removing more behavior from the cache for complexity reduciton
+ Cache is no longer contained in app structure
This should let us later throw this into an Arc<Mutex<>> for better
async support.
! Also more cache access safety is being done on the front end because the cahe doesn't guarantee much on access
Perhaps some convenience wrappers would make this look nicer(with inline)

!!! Main needs a lot of flattening
It's technically not much of a problem since most of the code behind 1/2 match's
which really just bloat the hell out of indentation making it look bad when its not _that_ bad.
However it still bugs me so I should probably do something about that(also (inline))
2021-04-14 22:44:57 -07:00
shockrah 39ae3ba6be Adding example of websocket url 2021-04-14 22:39:06 -07:00
shockrah 0b0bcd5de6 First pass docs for rtc capabilities 2021-04-14 22:38:43 -07:00
shockrah 42d6a77050 - Removing network control from the cache
From this point forward the cache should basically just be a data container.
Methods on the cache object are there for convenience and should only ever
return very simple data [booleans, integers, () etc.].
The cache should also have very litter/no concept of commands to decouple it
from the renderer process.
! list_channels is still implemented in the cache however it remaains a vestige
and will be moved to the net module

Also the client is getting some new packages for websocket connections and
while they work some additional work has to be put in to figure out where fug to
actually place the websocket so that it can update the cache+DOM in a seomewhat sensible fashion
2021-04-11 17:39:54 -07:00
shockrah 3708d2cd08 Removing websocket changes as they don't architecturally work and thus require too many workarounds
A better architecture must be implemented to have good websocket support
2021-04-10 17:09:25 -07:00
shockrah 102c3b2a10 Electron on Linux hangs when making two successive async calls over the network
Without this calls to /channels/list would break after the first API hit
2021-04-10 17:09:25 -07:00
shockrah 72f3461341 ! Due to user-id's being randomly generated it is now much more likely that id's can't be parsed by javascript properly
Json-bigint should help alleviate this issue by correctly parsing numbers for us

- Removing url method in favor of new url property in server config model

+ Building UserConfigs and ServerConfig from typical JS Obects
This should reduce some up front complexity regarding parameters

! Fix: using new url property instead of protocol + host + port

- Reducing upfront complexity of public channels API
* Also moving to using new updated server.url property
* Channels also handles its part of updating the dom now

bigint parsing in configs now

* More switches to server.url property
2021-04-10 17:09:21 -07:00
shockrah 89bd257213 * More explicit query string guard in /channels/list endpoint handler
* rtc::make_url now builds websocket urls more robustly incase we have to percent
encode them
2021-04-10 16:45:13 -07:00
shockrah 283c201e96 * More explicit query string guard in /channels/list endpoint handler
* rtc::make_url now builds websocket urls more robustly incase we have to percent
encode them
2021-04-10 16:44:24 -07:00
shockrah f92ced1241 + Websocket setup is almost proper but it completely hangs after we start listening for connections
Of course this is an issue because we can no longer get data from the user
A full rearchitecting may be in order here however its not impossible to modify the current architecture so far

The main issue here is that the cache is doing too much I think

If the termion(main input) task does its thing, and a socket task,
waits around waiting to build a sub task(socket) then we may be able to
pass messages to the socket task to open up sockets when request
This allows us to have shared state via message passing and a main task
can take care of updating the cache.

The renderer can then request data from the cache when it needs to render stuff
2021-04-09 22:10:08 -07:00
shockrah a0b8e82000 Adding tungstenite for async websocket support 2021-04-09 21:55:31 -07:00
shockrah abd2537918 really simple debugger logger, will remove at some point but it has use right now 2021-04-09 21:55:04 -07:00
shockrah bdd4a63a8d * Fixing url construction for websockets 2021-04-09 21:52:57 -07:00
shockrah 5df5329b6c * Fixed authentication parameter parsing for both users and servers
* Jwt is now given over the query string as many websocket libraries make
it hard to pass header parameters
Parsing these values should also work fine but we'll see in time
2021-04-09 18:59:10 -07:00
shockrah 7eb81f38f2 * Fixing bad /message/send route callback
Issue was that we weren't sending the correct headers/body data(ya I know)

- Removing mouse control from app since its completel irrelevant
!+ Putting app messagses behind a mutex for now since they are written to by
sockets and renderer process in the next few patches
2021-04-08 21:21:56 -07:00
shockrah 6088b0836f + Send message callback now implemented
+ channel switcher works as well, although it relies on /lc being ran prior to
its invocation
2021-04-08 15:45:20 -07:00
shockrah 9ad963f097 + Adding POST handler for message sending /message/send
Working as intended
2021-04-08 15:34:46 -07:00
shockrah fe448d07d4 Better naming scheme for when we start requesting voice channels after this 2021-04-07 22:17:28 -07:00
shockrah ceabc73cf7 Fixed hanging json-api issue with dev-run script 2021-04-07 21:06:46 -07:00
shockrah 368c9bc95f + Better logging in main
+
2021-04-07 21:06:08 -07:00
shockrah d030accddc * Adding id to insertion call in db-lib::channel::add
This is done at the driver level because the member id's are randomly generated
u64's
2021-04-07 20:48:58 -07:00
shockrah 86cd551309 Adding option to build release version of tui 2021-04-07 20:42:30 -07:00
shockrah 31dd85d8f0 simple make file to avoid repeatng args all the time 2021-04-07 20:41:54 -07:00
shockrah 704f372c3e + Adding list_channel handler
More agressive channel caching may be required here but it works as an MVP for now

* Switch to hashmap for container of channel caches instead of vector
This should help with cache accesses as we don't have to iterate anymore
Also it makes data fetches much cleaner/shorthanded

* net::list_channels has its own private structure for collecting channels vector
This is basically required and is in the scope of the func itsefl to not pollute any other namespaces
2021-04-07 20:38:06 -07:00
shockrah 8be5547d9e fixed channel api type 2021-04-07 20:24:31 -07:00
shockrah 10e8b7331f + Login route added to net module
! No wrapper goes around this module to avoid fragmentation
Note that basically every network hit is built around try's so they'll all return
HttpResult<T>'s
2021-04-05 17:41:28 -07:00
shockrah adea888b21 - Removing fluff from config structure 2021-04-05 17:37:43 -07:00
shockrah 0bee002164 + List channels backbone in place
Technically everything is there but I haven't really tested anything yet
Will 100% need fixing + optimizing

+ Switch server Command also working
! Requires a jwt check to avoid expensive+unnecssary network /login hits
! Probably have to hold onto the input to avoid spam somehow

ChannelCache is nearly ready for manipulation
2021-04-05 17:36:24 -07:00
shockrah 5ab2c961f6 - Adding listhost + listchan command variants 2021-04-05 17:00:59 -07:00
shockrah 939ee8c73f + Ignoring empty lines
+ Only adding / on '/' input activation
2021-04-05 16:54:27 -07:00
shockrah a8d4ca9028 Removing blocking client from reqwest dep as its not needed 2021-04-05 16:53:19 -07:00
shockrah 0bb7c5f49b adding / to start commands now 2021-04-05 14:24:21 -07:00
shockrah 3c10d342eb - Removing dead_code attr from TEXT_CHANNEL
+ Adding Clone derivation to Channel struct
2021-04-04 17:36:58 -07:00
shockrah a098121ab8 * Messages now grow from the top to the bottom(new messages ontop)
+ Option to enable/disable message-box borders

* Switching over to using bold! macro where applicable (especially in command.rs)
This change mostly reduces the eyesoreness of the termion code
2021-04-04 14:17:39 -07:00
shockrah 6fff48cfab * Messages now grow from the top to the bottom(new messages ontop)
+ Option to enable/disable message-box borders
* Switching over to using bold! macro where applicable
This change mostly reduces the eyesoreness of the termion code
2021-04-04 14:16:54 -07:00
shockrah 43e3f93aad + Handful of useful macros/functions that are used basically everywhere
! There's no real order to these as they are mostly miscellaneous
2021-04-04 14:15:06 -07:00
shockrah 1c85c88f96 * Normalized cache data structures a bit
* Also changed up the naming scheme to be more clear
* Config -> ConfigFile
- Removed ConfigFile::server_url method
! The ConfigFile struct is now pretty much entirely used for disk operations
* update_jwt method also take in correct parameters
2021-04-04 14:13:37 -07:00
shockrah 3f1dfbf824 Adding rtc notifications to relevant endpoints
! Still need a feature flag to compile this optionally instead of all the time
- Removed warning in main's auth::login_get_jwt call
2021-04-02 13:56:44 -07:00
shockrah be54ce6a57 - Removing Claim::new since its used once
+ New notify driver function which does the network magic we need
+ Adding event emitters for the following events
"delete-channel"
"create-channel"
"update-nick"
2021-04-02 13:55:21 -07:00
shockrah 17f281c91c Removing websocket library for tokio-tungstenite (rtc) 2021-04-02 12:29:53 -07:00
shockrah e496fbdd1a Message event now asynchonously emitted from /message/send to rtc server
+ Server claims are now also generated per request so this could get some benefit from caching the jwt
One idea is to cache a jwt for a temporary amount of time
+ event! macro also lets us generate json payloads more easily
! Still requiring this module to built under an optional feature however that should be done easily
2021-04-02 12:11:59 -07:00
shockrah c6a49a8437 User JWT's now have nbf field in claims set in seconds 2021-04-02 12:06:17 -07:00
shockrah 715f334619 - Removing peers module for simplicity and reduction of data complexity
Basically the peer map system that we had before isn't going to work simply because
succesful client connections already have the data we need inside them.
Adding a whole wrapper around this just complicates things in a way that doesn't
really give any benefit.

For now every message is echo'd to all connections but that is easy enough to change
2021-04-01 17:21:57 -07:00
shockrah d8171f8b03 + Replacing rust microservice with more lenient js built server
The JSON-API can't _really_ use regular http requests because this server then has to do a lot of
multi-threading nonsense.
For the sake of simplicity for myself and others that try to write their own FC
compliant servers: the rtc server(for now) only takes in websocket requests,
and attemptes to discern servers from users connections for event handling
2021-04-01 11:41:51 -07:00
shockrah afdeef0a49 More warning removals and more db-lib result wrapping
This should be everything now
2021-03-30 23:00:50 -07:00
shockrah 4615374357 Comfy dev scripts to make life a little bit easier 2021-03-30 22:49:46 -07:00
shockrah 498da4c899 Updating dependancies to latest versions and removing some fluff interim libs 2021-03-30 22:49:20 -07:00
shockrah c45f8d0482 More db-lib ok/err wrapping
Some misc changes that don't really belong with anything else are bundled here
Those changes aren't actually very important however
2021-03-30 22:46:59 -07:00
shockrah 75dcb7b73e No more jwt db checks
Also wrapping relevant db code with ok/err checks
2021-03-30 22:45:39 -07:00
shockrah 41202507c1 removing warning 2021-03-30 22:44:09 -07:00
shockrah 4cbbfdd7a2 mysql_async updates
* More shorthand usage for clearer code in invite create
2021-03-30 22:43:07 -07:00
shockrah 54f4dd53be - Removing non-result returns
+ Updating further for newest version of mysql_async
2021-03-30 22:42:34 -07:00
shockrah fea94bffa0 Removing features flag from db-lib as its not needed 2021-03-30 21:41:57 -07:00
shockrah 15b56353b8 Removing passthrough for message send handler
This means we get more opaque errors for the clients but it also results in way less code to maintain
2021-03-30 21:39:44 -07:00
shockrah d88948385d Upgrading mysql_async to latest version
This version comes with the benefit of having much more concise exec_map functionality among other things

This commit and the following ones also move db-lib to using a more 'try' based approach
2021-03-30 21:38:16 -07:00
shockrah f7d90e4a09 - Removal of auth code in db-lib 2021-03-30 21:36:21 -07:00
shockrah 8eec9abd74 + More dev files 2021-03-30 12:25:34 -07:00
shockrah 56e4e22b4c + JWT Authentication
* Server JWT's and Client JWT's built with seperate signature types
2021-03-30 12:24:10 -07:00
shockrah 5bbc57313f + Json-API now pushes proper new-message event to rtc server correctly
+ Rtc server takes in new-message correctly but with basically no ux/auth for now
This should be fine as we can layer on security afterwards with no issue
2021-03-25 20:46:10 -07:00
shockrah 291505201b removng fluff and adding link to repo 2021-03-24 15:19:43 -07:00
shockrah 52fab01119 + Clearer logging from the json api
* Renamed db helper function to be (debatably) more clear in its intentions
	That function and everything in that module is quickly become awful however

+ Adding some notes for later db-lib development
2021-03-24 01:43:52 -07:00
shockrah a1f86fdf6e * Moving away from auto_increment
! Initial id size will stay as u64 but u128 are easily the next major change
This change shouldn't break clients written in high level languages since most already use 128's under the hood anyway
- This commit also removes the auto_increment flag from basically everything that uses RNG id's
2021-03-24 01:14:37 -07:00
shockrah d02084a22c + Adding proper file upload support
! Requirements for variable message fetching not yet met
! /message/get?id=<id> needs a way of passing back file contents
! The file upload is sketchy at best and not necessarily guaranteed to sync the
database with the file system
2021-03-23 17:16:05 -07:00
shockrah 92dc4d888f meta tags and favicon 2021-03-22 22:39:51 -07:00
shockrah 185b64d9db Base version of the rtc server
Authorization is required for implementation
So is a proper messaging model to make implementation clean and scalable
For now this serves as a good starting point for the rest of this project
2021-03-22 02:45:54 -07:00
shockrah 3ee22ff932 - Removed remaining mentions of joindate 2021-03-20 22:54:00 -07:00
shockrah 975acfd606 Backend REST API changes:
* Fixed weird match with get_online_members
+ Now using special(smoll) Public Member struct for public member data fetches as.
They're size on networks should be pretty small so we can package a ton of them in a single request

DOCS changes:
- Removing references to unix timestamps in seconds
- Removing references to joindate
* Exact content-type values now specified
2021-03-20 22:44:22 -07:00
shockrah 03c41b6833 - Removed joindate from members schema
+ Auth module now uses std::time for time based calculations
! All time notations are i64 and rounded to the Millisecond

* Moving db pool to a lazy static to avoid constructing a whole pool on every request

+ Adding more logging per request, even if its lazy logging
* Content-Types are now correctly written per type
2021-03-20 19:47:59 -07:00
shockrah 8812ff7198 + text/plain Message sending works now
+ More skeleton code for the caching system

! Ready for rtc on text based message listening which is where heavy caching comes into play
2021-03-20 19:35:38 -07:00
shockrah 530d287431 Merge branch 'master' of gitlab.com:shockrah/freechat 2021-03-20 15:00:58 -07:00
shockrah 939042a4c5 skeleton code ready to start pluggin in async fetch calls to remote servers
Some model still has to be built to put RTC somewhere
Also stdout still requires async handling
2021-03-20 15:00:49 -07:00
shockrah 988aa9f155 Removal of chrono and async_trait as dependancy in db-lib
Removig chrono from api code as well

Removing chrono as dep in api code
+ Using Content-Type for /message/send content type

Updating cargo lists for removal of fluff deps

Removal of more fluff

Addking makefile to avoid compiling debug builds by accident while developing
2021-03-20 14:59:06 -07:00
shockrah ba14600b34 Removal of chrono and async_trait as dependancy in db-lib
Removig chrono from api code as well
2021-03-20 13:56:02 -07:00
shockrah cb5975f235 FULL REWORK AND PORT TO TERMION
This patch is simply meant to mark the beginning of the newest phase for the tui build.
I've 100% settled on using termion as the backend and marking what is done so far

Renderer side:
	Termion has a similar issue where moving data happens very quietly so its best
	if the two (renderer and cache) have the their data to use as they please

Cache Side:
	Basically we own the data we're using because we constantly have to mutate data
	ourselves

Config in the middle:
	Mutable but only from the rendering side because the cache is completely transient
	It technically to own its data but it does anyway because the render(backend)
	 likes to consume data like there's no tomorrow
2021-03-17 20:39:42 -07:00
shockrah c3d5c75cc0 messages rebind now work properly and sends message correctly,
Message constructor no longer incorrectly check message type
2021-03-12 23:16:06 -08:00
shockrah c667f69d0e Fixing busted config adder thingy 2021-03-12 21:36:44 -08:00
shockrah 67575cda53 adding tsc to make build 2021-03-12 02:20:33 -08:00
shockrah 5b2e92da06 Caching is still being designed but it _will_ be written in typescript to a more well docc'ed api for everything else
+ Basically im doing this to straight up abuse
- Moving cache types + methods to their own module
2021-03-12 02:20:00 -08:00
shockrah 9ce04e96a7 Query string parameter 'type' is now enforced by the api
+ Flag is literally called 'type'
2021-03-12 02:17:06 -08:00
shockrah dc117ba02f Moving message type to types.ts 2021-03-11 17:22:22 -08:00