08-22-2016, 12:37 AM
Good day community.
Today we are glad to announce one of Limelights biggest release.
Before I continue, I would like to thank the entire staff-team for their trust in me and my work.
To the community:
Thank you for your patience during the year without a dupe-manager.
You may wonder what *what the hell took so long?*
Well. Here is the answer:
On a serious note:
The UCP is based on my web-framework called "Cloud2Box - Community Framework".
My goal was to have a product in stock for my company called "Cloud2Box".
Because every company needs a product that makes them known for it.
For example, at Limelight it is the CityRP 2 gamemode.
Since I'm familiar with programming languages for about 9 years, I decided to make something special.
Something that could be used as a base for bigger projects.
The UCP was born.
Developer vision/journey as to why I wanted something special for my company and Limelight:
So one year ago I decided to work on a user control panel. It was clear to me that this application must be top-of-the-edge, using the latest web-technologies, top speed and an extremendous UI (user-interface). Since then I had worked on a framework for my own company here in Germany. I wanted to develop a product which can be used as a base for bigger ones.
The most important things that popped into my mind:
- DRY = Do not repeat yourself. Code should be fully reusable
- Modularization: The framework should be modular so that parts could be extracted and used elsewhere
- Speed: There shouldn't be any memory/cpu overhead when dealing with thousands of connections (non-blocking I/O)
- Security: The framework should include trusted and latest security components, applied to frontend and every backend
- Indepently: The framework should communicate with other products
After I gathered all the information, I started to write down all things that I was going to need.
Since I've taken security very serious, I didn't want to have the UCP run on the gameservers dedicated server.
It was clear to me that I need to access the filesystem of that dedicated server, without getting heavy traffic overhead.
Also as stated above, security and speed was a top priority.
The previous version of the dupe transfer is flawed with critical security bugs , yet it uses the FTP protocol for interaction with the filesystem. It also opens multiple FTP connections for every single action such as listing (LIST), storing STOR).
In addition, a hardcoded system user is needed to be set as FTP user which then has access to the files.
This is something I've highly disliked since it requires the script to have that user exist at the system level.
After a few days of planing it was pretty clear that I needed to get around that protocol.
Now at that time a lot of other file protocols existed but fairly I did not really liked any of them. They either were too complex or just not suitable for the use-case. I also wanted an option to combine multiple hosts into one, basically a cluster system.
Soon I realized that I had to develop my own protocol or an application on-top of an existing protocol.
We all know that the HTTP protocol is widely used among the net, thus I decided to build an API (application interface) with support of that protocol.
The journey started.
Component 1: The API-Framework (Sys-API) *Open Source*
Back in February 2015 I started to develop my own API-framework called sys-api. It is an API-framework based on another API framework called RestifyJS.
After two month of development the API was ready for its first release on Github. On May 9th 2015, I released it - knowing that I was going to improve it over couple of months. During the months the API became stable and was ready to be used in a production environment.
Today it offers multiple features such as authorization (+ bcrypt-handler), CORS, TLS, body-parser, accept-parser, date-parser, query-parser, jsonp, gzip, throttle, conditional-request, extensive routing engine, Logging, plugins and addons.
Source code: https://github.com/Cloud2Box/sys-api/
Component 2: The Storage-API (Sys-API instance) *Closed Source*
The storage-API is an instance of the Sys-API. It contains all the logic to perform operations on the storage filesystem.
It is seperated into frontend and backend. While the frontend handles all the requests/responses, authentication, validation and verification, the backend runs tasks.
Component 3: The Framework (Cloud2Box Community Framework) *Closed Source*
In July 2015 I started working on the framework with primary focus on the points described above.
The framework uses the latest technology such as NodeJS and AngularJS - just to include two of them.
The framework itself can be seperated into three parts:
+ Component 3.1: The clients frontend [ Templates, JS Interactions ]
+ Component 3.2: The clients backend [ Controllers, Directives, Services, Filters ]
+ Component 3.3: The applications backend [ Repositories, Providers, Interfaces (mysql, mongodb, sys-api, cache-server), Routing, Session-Handling, View-Rendering, Localization, Authentication (basic, oauth, openid), Filters, Validation, Tests
Each component is seperated into sub-components, which can be used on their on. Thus making it highly re-usable, modular and indepentent.
Component 4: The Sys-API interface (node-sysapi) for component #3.3 *Closed Source*
In order to have component #3.3(Application-backend) communicate with component #2(Storage-API), a seperate interface was needed. After a few days the interface was ready and attached. Later then I added multihost support to combine multiple hosts into a cluster.
Component 5: The capture-engine (fs-capture) for component #2 *Open Source*
Whilst improving component #2(Storage-API), I seperated various filesystem operations into one component.
fs-capture finds directories and files based on a specified target. It is a capture-engine with a promised asynchronous control-flow.
It is extremely vital for component #2 in order to get the desired results.
In this case component #2(Storage-API) uses fs-capture to get a dupe and/or directory based on given input.
Source code: https://github.com/Cloud2Box/fs-capture/
Here is a simple overview of the overall systems:
As you can see the client does not have direct access to the storage. Every request to the storage/response of the storage goes through the Server backend. The storage itself is kept offline and is not accessible from the public scope.
Server backend and API have various security options in place, such as authentication, validation and verification.
Please note this is not the full structure. Every single part contains other subparts which I won't expose due to security reasons.
Early Developer versions:
Heatbeat procedure between Storage-API and UCP Server-backend:
^ All the paths such as "/var/www" and ports are not the same as in production mode. That's why I can freely expose them here. In addtition, the storage-API is not accessible from public space. (check explanation above)
THE USER CONTROL PANEL
+ The UCP is restricted to users/staff of Limelight. Players that haven't played on the server yet, are not allowed to log in. A friendly notification is shown when attempted.
+ Banned players are not allowed to log-in. If a session exists, it'll be destroyed.
+ Two versions of the UCP are available:
MASTER [live version] and RC [release candidate].
The master version is available to everyone whereas the rc version is restricted to testers. The rc version contains bug-fixes and new features that will be merged into the master version, once tested/ready. Hotfixes will be applied to both versions.
The statistics page gives you an overview of various metrics.
Get an insight of various totals.
Player(s) of the month(s):
Two players are chosen based on their activity during the month.
Want to know the previous ones? No problem. Simply select a month:
Most active (based on total hours):
This list contains the most active players based on their accumulative amount of hours.
You can also list up to 100 entries at once:
This list contains the richest players.
As with the other metrics, you can also list up to 100 entries at once.
This list contains the most supportive players which spent a lot of 'real' money.
(Don't want your name in it? Contact me via private message and I'll blacklist it)
As with the other metrics, you can list up to 10 entries at once.
Highest REP count:
This list contains the most awarded players.
As with the other metrics, you can list up to 50 entries at once.
TECHNICAL POINT OF VIEW
Stats are retrived from a seperate cache-server. They are kept in cache for 30 minutes until wiped.
If you visit the stats page but nothing is in cache, you will be the special one who initializes the caching-procedure.
Then fresh stats will be fetched, processed and put into the cache-server for 30 minutes.
^ There is no need for your to reload that page. It'll automatically render all the stats once they're avialable in cache. It might take up to 10 seconds.
The charcter page lists some OOC and in-character information.
For now there is not much. Just your pic, name, rank and steam-ids.
Some interesting stats about your in-game character.
Wanna know your latest transactions? No problem:
You can check your jail-status as well:
The membership tab lists every information about your membership status.
Want to know when it expires or how much you've spent so far?
No problem, we got you covered:
Not a member yet? You're welcome!
Are you a staff member? Well then:
The dupe manager is a fully featured application, giving you the most comfortable and fastest way to manage your contraptions.
Drag 'n Drop support:
Simply drag a dupe/folder and drop it into a target directory.
Copy mode [added in version 0.8.3]:
As of version 0.8.3 you can copy a dupe/folder. Means that the original file/folder is not moved entirely.
If you want to move a dupe/folder instead, simply disable copy mode.
Copy mode is enabled per default.
Want to get a better/styleless overview?
The spectator mode got you covered:
Yup. Due to the lovely Sys-API interface (component #4), the dupe-manager can retrive/send folders/files from/to multiple hosts.
Want to know what's inside your dupe? No problemo.
Simply right click on it and you're good to go:
[Updated with version 0.9.8f]:
Read more: http://limelightgaming.net/forums/thread...#pid131409
Easy. You can rename dupes and folders (except for home).
Easy. You can delete dupes and folders (except for home).
[Updated with version 0.9.8f]:
Read more: http://limelightgaming.net/forums/thread...#pid131409
Did I tell you that this is possible?
The entire communication is encrypted.
Special thanks to Faustie for sorting the Cloudflare issues
(This post was last modified: 12-21-2016, 01:21 PM by Burnett.)
The following 42 users Like Burnett's post:
• Bambo, Blurr, Brax, Brennan, Brynn, Cameron, Cooli, Cunix, Faustie, General Rickets, George, Innovative, James, Jompe, Jono, Kpred, L=I²=Am³, Liv, Lods, Lord Octagon, Markus, Mr.Sir, MrMeme, Nacreas, Nightmare, Noble, Nudelsalat im Panzer, Obay, Overlewd, Pinky, Project, RK-FIN, Rocket, Safira, SirWulf, Sours, Temar, The Rock, TheBlackShadowOfGod, Toxic, Venom, Zeyon