-
week12
- 4 August
Twelfth Week: World Map, Blacklist/Whitelist, Frontend Console
This week, I decided to work less because I already finished all of my normal goals. There are still some stretch goals to do.
Resource for Console Help Text
Merged.
Unit Tests
All of the classes that can reasonably be tested have been finished.
As for the documentation for new features, I already did it all while I wrote the classes. As said before, all tasks have been done, although I still have some stretch goals to work on. I plan on writing an user’s guide or something similar at some point. There are a few things with the UX on the frontend that I would like to change. Other than that, in the next weeks I could work on some of my stretch goals. -
week11
- 28 July
Eleventh Week: World Map, Blacklist/Whitelist, Frontend Console
This week was mostly full of personal stuff that prevented me from doing much work.
Add Server Facade to Omega Distribution
Since the jopenvr issue that was blocking progress on this was resolved last week, this was merged.
Backend for Blacklist and Whitelist
No changes, but merged.
Frontend for Blacklist and Whitelist
After the redesign of the prompt last week, the challenge switched to trying to refresh the prompt. I found a solution to the problem by closing and reopening the prompt instantly. This worked, but it wouldn’t refresh for other clients when something was changed. My mentor, Gianluca Nitii, figured out a way to refresh the prompt from the server by making a function variable in the user management tab’s state that can be called by the user management tab model. The blacklist/whitelist prompt sets the state of the tab so that the function variable is a refresh function in the prompt.
Make the Frontend Execute Server Commands Only
Ultimately it was decided that a server-only solution would work better, so this was closed.
Resource for Console Help Text
Instead of just being used for the help text, this PR now does everything for the frontend. The work done here is mostly copy-pasting code from the frontend to the backend under the console resource and translating it into Java.
Unit Tests
This PR adds unit tests, so far only the system resource is done.
Again, I didn’t have much time to do work, which is why there isn’t much for this week. Next week is the last week in GSoC, so I want to finish adding unit tests and documentation. I will add unit tests to all new features I added over GSoC, and possibly other things if it looks like they are needed. As for documentation, I already fully documented all of the new code I added, so I might add documentation for end-users to reference. If I finish these completely in the next week, I will work on the stretch goal of adding a config system to the facade server backend. -
week10
- 21 July
Tenth Week: World Map, Blacklist/Whitelist, Frontend Console
This week was about finishing up the world map resource and getting started on adding blacklist and whitelist support to the server facade.
Backend for the World Map
The only changes this week were for destroying the entity that was created to load the map and adding an admin check for loading the map. The first was done by getting the entity ref used to load the blocks as a return value, then destroying it once the loading is done. The admin check
Frontend for the World Map
No changes, but this was merged.
Backend for Blacklist and Whitelist
The only change here was to callnotifyChangedForAllClients()whenever the blacklist or whitelist was changed. This makes it so that the frontend will automatically update the blacklist/whitelist state whenever it gets changed.
Frontend for Blacklist and Whitelist
I decided that the previous input prompt didn’t make sense with 3 pickers, as it would be too confusing for the user and too difficult to code. Instead, the prompt now displays the ids of all players online, on the blacklist, and on the whitelist, along with a text input for the id to modify. The list of players is intended to be copy-pasted from. However, the prompt does not quite seem to refresh properly. Whenever a player is added to the blacklist or whitelist from the prompt, it does not update on its own. It does update correctly after any of the buttons are clicked again. It may be better to just move it from a prompt into a tab, as that would make refreshing easy. Also, in testing this I found a bug with the blacklist code that I wrote for the engine. If a player is blacklisted and exits the game, then the server doesn’t correctly remove them from the game and they still appear to be connected to the NetworkSystem.
Make the Frontend Execute Server Commands Only
To make the frontend UX better, this PR changes the console to make it so that only commands that run on the server can be executed. The Console resource’s GET method added earlier for the autocomplete was used as a list of acceptable commands. If a command is entered and it isn’t in the list of commands, then it cannot be executed. If the command entered ishelp, then the help command isn’t actually run, instead a help message from the backend resource below is manually added to the console.
Resource for Console Help Text
The frontend needs a way to get the filtered help text, so this PR adds a resource for it. Essentially, this resource uses a stream to filter commands that don’t run on the server usingConsoleCommand.isRunOnServer(), then it emulates thehelp()method inCoreCommands.javafor each command in the filtered list.
Add Jopenvr to the classpath
This small PR adds jopenvr.jar to the server facade’s jar classpath. This should make it so that the server facade can finally run correctly in the frontend.
Next Week, I should be able to finish the blacklist/whitelist implementation on the frontend. It shouldn’t take more than a few hours to do. After that, I plan to write unit tests for all of the newResourceclasses I added over the past few months, as well as anything else in the backend that looks like it could use a unit test. At the same time, I want to work on the stretch goal of being able to access the local files of the server from the frontend and possibly make them modifiable as well. -
week9
- 14 July
Ninth Week: World Map and Blacklist/Whitelist
This week was about finishing up the world map resource and getting started on adding blacklist and whitelist support to the server facade.
Backend for the World Map
The work done this week made it so that chunks could be loaded from the frontend without having a player in-game. This was done by creating two components, LocationComponent and RelevanceRegionComponent, and creating an entity with them that then automatically generates chunks. The LocationComponent is set to the center of the requested map to serve as the center of where the chunks are loaded, and the RelevanceRegionComponent is set to cover enough chunks so that there will never be any unloaded chunks. It takes some time for the chunks to be loaded, soThread.sleep(1000)is called whenever theWorldProvidergets an unloaded block. This makes it poll once per second to minimize performance impact. The only thing left to do for this PR is to look at improving code readability by naming constants and looking at removing the entity after it serves its purpose to avoid memory leaks.
Frontend for the World Map
Most of the new work here is for UX. The error message on startup was fixed, and the world map input variables now stay the same after submit is pressed, which is much more intuitive. The y-coordinate input is now only rendered if the surface checkbox is disabled, as there is no reason to have both at the same time. The layout of the buttons was also improved.
Backend for Blacklist and Whitelist
This backend PR is relatively simple, consisting only of adding a new resource class that is very similar to the server admins resource. There are actually two different resources created from the class by using an enum. All the resource really does is check if the resource is a blacklist or whitelist, and then it adds, removes, or gets the list, depending on the http method.
Frontend for Blacklist and Whitelist
The frontend for the blacklist/whitelist is more complicated, involving adding a new prompt with multiple pickers. I am not entirely sure if the prompt actually works yet. The prompt gets the online players and the current blacklist/whitelisted players and allows an admin to add/remove any of the above players from the blacklist/whitelist. Aside from the prompt, there are some new functions for querying the REST API for the new resources.
For next week, the goal is to finish up blacklist and whitelist support by making the POST and DELETE methods work correctly. After that is finished, I will work on modifying the console so that only the commands that work on the server are visible and can be executed. If possible, I plan on modifying what gets displayed by executing thehelpcommand, possibly on the frontend. If that gets finished, I will then work on adding more unit tests, starting with the system resource added in week 2. -
week8
- 7 July
Eighth Week: World Map
Work this week consisted of improving the world map. Additions include modifying the brightness of certain parts of the texture depending on a block’s y coordinate, changing the way of sending the image data over REST to be base64 encoded, and bug fixing/testing.
Backend for the World Map
The first new commit here refactored the WorldMapResource class to have all the functionality of WorldMapMetadata, and WorldMapMetadata was deleted. It also drastically improved the performance by creating the image on the backend instead of the frontend. Instead of sending an array of blocks and a lookup table for the colors for the frontend to display, the backend just creates a BufferedImage instead. This BufferedImage uses the colors it obtained from the blocks to build an image with a bunch of rectangles, then the BufferedImage is encoded into a base64 string and sent across the REST API to be displayed by the frontend. The second commit changed the getRGB function into a function that also takes the y-coordinate into account to darken the colors, where lower is darker.
Frontend for the World Map
The changes here were made to keep the frontend working correctly with the backend. More specifically, the display area of the world map image is now just an RX.Image tag, where the source is the base64 encoded image.
Here is an image of the world with the center at (0, 0, 0) and a width/height of 250.
For next week, the goal is to finish up the world map by adding some way to load the blocks in the world from the frontend without having to connect to the game with a normal client. After that is done, if any bugs are present then they will be worked on. It is likely that these tasks will be done before the week is over, and I will work on adding server blacklist/whitelist support to the frontend. I already added support for these to the engine before GSoC started, so it should be almost trivial to add some new buttons to the user management tab with this functionality, as long as the blacklist/whitelist code works correctly. -
week7
- 30 June
Seventh Week: World Map and Packaging
Work this week consisted of conceptualizing and executing the world map and working on packaging. For packaging, I opened what should be the last PR to change the Omega distribution. I also made a working version of the world map resource, which displays a part of the game world.
Modifying the ServerFacade Jar
It turns out that I put the launch scripts folder into the wrong directory in last week’s packaging PR. This PR moved the launch scripts into src/main/resources so that they would be included in the jar.
Change the Omega distribution to contain the server facade
Changes in this PR modify the Omega distribution with an additional property and some gradle code to declare the server facade as a dependency. The server facade is declared as a dependency so that it and all its dependencies are automatically downloaded when trying to build the Omega distribution. Aside from that, another part of the code takes all files in the server facade dependency, then filters out anything that isn’t in the launch scripts directory. The result of this is that the contents of the launchScripts directory are copied into the omega distribution.
Backend for the World Map
The current setup for this uses a center location to search from along with a block length and width to display a map. A 2D array is sent across the REST API, containing the names of all blocks found through the WorldProvider class. This uses the coordinates provided from the frontend. Optionally, the y coordinate of the surface can be found using methods similar to the how the Minimap module does it. The texture for each block is then found, and then the rgb value of each pixel in the block is averaged to give a color. This color of the block is then matched to the name of the block and sent across as a look-up table for the frontend to use in displaying. However, there is currently a big problem with how the blocks are obtained in the world. The blocks have to be loaded to get any data on them, which means that there has to be someone connected to the game with a regular client to load the blocks so that we can get the names and textures of the blocks.
Frontend for the World Map
The frontend is used to send information to the server on where to display the map. Once the data is processed on the server, the server then gives data to build the map. Right now, the colors are rendered as a 2D array of ReactXP ImageSVGs that are squares with a single fill color. This works, but it is really inefficient and causes a lot of lag.
Now for some screenshots:
This shows the world map at y coordinate 35 compared to the real game.
This is the same area where the y coordinate is always at the surface, more like a minimap. The image might actually be incorrect due to a bug.
Next week will consist of more progress on the world map resource. My goal is to add a sort of brightness level for blocks that are on different elevations. There is already an implementation of this in the Minimap module that I can copy. -
week6
- 23 June
Sixth Week: Admin Permissions, Packaging, and World Map
Work this week consisted of finishing admin permissions, opening a PR for changing the build.gradle file and some of the logic for KeyStores, and experimenting with adding a world map resource for viewing the game within the frontend. Backend for Admin Permissions
This week consisted of a bit more refactoring to better encapsulate the admin permission system with existing classes. All of the bugs easily found through testing were fixed as well. Frontend for Admin Permissions
No new work was done on this PR, but it was merged this week. Modifying Gradle and ServerMain for Packaging
The changes in this PR modify the jar created with gradle (which is the one that Jenkins hosts and can also be downloaded as a dependency). ServerMain was changed to accommodate how the KeyStore file used in https connections cannot be used because it is in the jar file for Omega distributions. There are also three new launch scripts, one for each major operating system. Only the Windows script has been tested, but the other ones might work. As for next week, the focus should be on finishing up packaging and adding a resource that gives information on the game world. The only thing I still need to figure out is how to copy just the launch scripts from the FacadeServer jar into the omega distribution. This should be relatively simple, only requiring something along the lines of this:As for the world map, the original goal was to use the AWT Facade to display the world by connecting like any other client would. I originally intended to somehow get the AWT facade as a whole into the web interface using technologies like GWT or VertxUI. However, after a bit of testing, I found out that it might just be easier to make a custom system using a resource that returns information on the world, then using that information to display something on the frontend. The AWT Facade in its current state needs some fixing up to be viable for this task, but the amount of work needed for that might exceed the amount of work for making a new system. I plan on adding a resource that returns information on some blocks in the world, most importantly the color of the blocks, the location of the blocks, and the display name of the blocks. The last two are easy to do, but finding the color has proven to be fairly difficult. Once all the data in the backend has been collected, it will be trivial to serialize it for the REST API. There will, of course, be challenges in this approach. I still need to figure out how to load the blocks before trying to get them, otherwise they appear as unloaded. There will also surely be other problems that I cannot think of at the moment. On the frontend, the goal is to display a top-down view of the world using the data given by the backend. There are components like ImageSVG or the regular Image which could be used to display things like RGB values.if (project.hasProperty('useFacadeServer') && project.property('useFacadeServer') == "true") { from configurations.facades include "run_facade_server_windows.bat" include "run_facade_server_mac.command" include "run_facade_server_linux.sh" into "" } -
week5
- 16 June
Summary of the first month:
week 1
Logout Button
Adds a logout button to the frontend. Currently only refreshes the page. Also, a lot of packaging stuff has been done over the weeks, starting from week 1.week 2
System Status Backend
System Status Frontend
Adds information on the system running the server to the REST API, including CPU usage, RAM usage, and server uptime.
Javadoc for the Backend (ongoing)
Adds a bunch of Javadoc to the backend code so that it is easier to work with.week 3
Console Autocomplete Backend
Console Autocomplete Frontend
Allows pressing tab to autocomplete commands for the frontend.week 4
Backend for Admin Permissions
Frontend for Admin Permission
Implements a new system where some admins can be restricted from certain actions.Fifth Week: Admin Permissions, Packaging, and First Evaluation
Work this week was mostly on admin permissions and a bit of work put into packaging. Backend for Admin Permissions
This PR, opened late last week, is now close to being finished, with some refactoring close to completion. It blocks admins from performing actions using the serverAdminPermissions.json file:
With the recent refactoring, admin permissions are integrated into the rest of the system, where individual resources can specify what permissions they require. Frontend for Admin Permissions
Coming from last week, the changes done have finalized the logic required for the frontend. All that has been done is simply creating a UI and sending/receiving data relating to admin permissions through the REST API. As for packaging, almost everything seems to have been figured out. I have managed to successfully run the server within the Omega distribution after manually copying some files in. These files could probably be added in automatically without too much difficulty. First, the facade server jar needed to be changed to include the classpath:build.gradle (under Server Facade)This change makes the jar read in the libraries of the server facade. To run the facade in the Omega distribution, I made a batch file (for Windows only) that runs the facade and stores the saves in the correct directory:jar { manifest { attributes("Main-Class" : mainClassName) attributes("Class-Path" : configurations.runtime.collect { it.getName() }.join(" ")) } }
java -Xms128m -Xmx1536m -jar libs/FacadeServer-1.0.0-SNAPSHOT.jar -homedir=”C:\Users\PCUser4\Saved Games\Terasology\server” REM Alternatively use our Launcher from: https://github.com/MovingBlocks/TerasologyLauncher/releasesconfigurations { // … facades { description = ‘Facades with dependencies’ } } dependencies { // … if (project.hasProperty(‘useFacadeServer’) && project.property(‘useFacadeServer’) == “true”) { logger.lifecycle(“Extra facade: Server”) dependencies { facades(group: ‘org.terasology.web’, name: ‘FacadeServer’, version: ‘+’) } } else { logger.info “Not using Facade Server” } } task distroPC (type: Zip) { description = “Adds modules to a PC distribution of the game. Exact modules are declared in a distro dir like iota” // … // We also want to add the facades given by the gradle.properties file from(configurations.facades) { into “libs” } }At the moment, there is still a bug, where running the server creates an empty rendering config file in the game directory. To copy in the files, I have experimented with the distros.gradle file used to make the Omega distribution: ```distros.gradle```extraModules=AdditionalFruits,AdditionalItemPipes,AdditionalRails… useFacadeServer=true ``` With all this said, the Facade Server and all its dependencies increase the size of the Omega distribution by about 30MB. Maybe it would be better to create a new distribution for this instead of changing the Omega distribution.To accompany this, there is a change in the gradle.properties for the omega distribution: ```gradle.properties``` -
week4
- 9 June
Fourth Week: Frontend for Console Commands, Admin Permissions
This week consisted entirely of working on the frontend and backend for console commands and admin permissions. Frontend for Console Commands
These commands allow for easier user management than manually using the console. Unfortunately, the way most console commands are programmed means that there needs to be a client connected to the game for execution. The only commands that work on the server are user management commands. Backend for Admin Permissions
A new file, similar to the already existing serverAdmins.json, holds the permissions for admins. The following resources were added to the REST API with this PR:- resources/serverAdminPermissions; GET gives the contents of serverAdminPermissions.json
- resources/serverAdmins/{adminId}/permissions; GET gives the permissions in JSON form of the admin’s permissions, PATCH changes the permissions.
The admin permissions system makes it so that some admins are blocked from taking certain actions.
Here we can see a list of permissions. If an admin tries to access a command and they are not allowed to, then the server will give them an access forbidden message. At the moment, the actual permissions are hardcoded, and most of the logic preventing access is hardcoded as well.
The individual permissions for each admin can be seen here.
This frontend change is mostly just to allow a place to change the permissions. At the moment, the only thing done is the UI and getting the data into a variable on the frontend, while the actual logic is currenly in progress. Next week is a buffer week that will consist of finishing up all the tasks for month 1. Admin permissions should be done early in the week. Hopefully packaging will be completed by then, and ideally there will be enough documentation. Another goal will be to update the version of ReactXP so that we can use the requestFocus() method to keep focus on the console command execution input. -
week3
- 2 June
Third Week: Autocomplete, Gradle, and Packaging
This week consisted of adding autocomplete to the console on the frontend and a lot of learning gradle to try to package the Server Facade into the omega distribution. Code for console autocomplete backend PR
This PR is very simple and simply adds a GET request to the console resource that gives all commands available to the server. This is needed for the frontend. Code for the console autocomplete frontend PR
The console autocomplete code is fairly simple, and upon pressing tab in the console command execution box, the console commands cycle through a list of matches. Packaging has proven to be more difficult than expected. Before this week I had basically zero experience with gradle, so I spent a while learning about it. Almost all of this week was spent on trying to do the packaging. I have spent a lot of the time looking at the PC facade gradle build script and the distros.gradle script used to make the Omega distribution. At the moment, I have managed to get the server facade to run in the Omega distribution, but I did so with some manual manipulation of files, which needs to be accomplished automatically with gradle. For next week, I plan to add a frontend for some of the more common console commands. The commands that deal with permissions for the console, kicking an user, and renaming an user could go into the serverAdmins tab, which means it could be renamed the user management tab. Another goal for the next week is to control the levels of admin permission on the facade server frontend, allowing some people to do more or less things. For example, maybe some people should only have the authority to kick users, but not ban users or change games. -
week2
- 26 May
Second Week: First Addition to the Backend/Frontend, Documentation, Launcher Packaging, and Autocomplete
This week was the first full week of work. Most of the week was dedicated to getting the system status on the REST API and on the frontend, while some other time was also put into adding documentation and looking into packaging with the launcher. Code for the backend PR
In the meeting on Sunday, it was ultimately decided that SIGAR was not the best choice due to it requiring additional natives. Instead, we opted for Oshi, which gives what we need without requiring any external additions. Oshi does have a dependency in JNA, but the engine already requires JNA, so nothing needs to be added. The version of JNA used by the engine is outdated, so I had to use a somewhat outdated version of Oshi. In the end, the information transmitted over REST consists of CPU usage, memory usage, and memory max. By using the built in Runtime and RuntimeMXBean classes, the REST API also receives the jvm memory usage/maximum as well as the jvm uptime (meaning server uptime). Code for the frontend PR
The frontend implementation for system status is fairly simple. Most of it is just displaying and formatting text from the backend. The other parts of the code are mostly just fitting it into the abstractions. I am also adding bits of javadoc to the backend classes when I have some spare time. The javadoc might be done before the next week, and a PR should come up soon. Console command autocomplete is currently in progress for next week, and should be done before the week ends. If there is time, some work will be put into packaging, assuming that the Jenkins build for the server facade works by then.
The approach that I plan on taking for autocomplete starts with exposing a list of console commands through a GET request:
(the text might be a bit too small, but it is a list of all available console commands.)
I have already gotten this data across to the frontend, and now my goal is to emulate the relevant parts of CyclingTabCompletionEngine to make the console commands. The frontend is written in typescript while the tab completion code is in Java, so I will have to do a bit of work converting it. For packaging, the first goal will be to add an option to the launcher settings allowing running the game with the facade server.
We need to be able to download the server facade from a source (which will be Jenkins for now) and then run it. Once the Jenkins build starts working, I will begin work on the launcher to make it work. -
week1
- 19 May
First Week: Frontend Changes, Packaging, and SIGAR
Most of this week was dedicated to final exams and learning the codebase for the FacadeServer and FacadeServerFrontend. Monday and Tuesday of this week were occupied by final exams. I Got to learn more about the design of the facadeserver as well, so future weeks will definitely be more productive. Packaging was a goal for this week, but there have been issues with making binary distributions of the server facade with jenkins, so this goal is put on hold for a short amount of time. I have played around a bit with the launcher, and I think I have a good idea of what to do to make the FacadeServer easy to download. Now that the first week is over and I have more knowledge of what I am working with, I should be able work much quicker. Code this week consists of two main things worked on. Code for the logout button
I added a logout button to the frontend. This button currently just refreshes the page, which both logs the user out and also disconnects them from the server. Ideally, it should only log the user out. Also, the native build for Android (and iOS too) appears to not work correctly. There is no logout button for it at the moment.
It appears that the memory percentage reported is slightly lower with SIGAR than with task manager, but it is a minimal difference. Maybe the difference is caused by the server being on a local machine. Code for this isn’t yet in any PR, but there should be one on Sunday or Monday. SIGAR makes it really easy to get a lot of data about the system. This is all you need for the CPU usage:
The original proposal only mentioned CPU and RAM usage, but it looks like it is easy to do more. Hopefully I won’t go crazy with adding too much! One concern with using SIGAR is that is requires a binary dll (or equivalent for other operating systems) to be in the path environment variable. SIGAR uses the Apache license 2, which allows redistributing binaries (I think?). It shouldn’t be too hard to add the dlls to the path with a bit of experimenting, but the hard part will be to find where in the code to do so.Sigar sigar = new Sigar(); try { CpuPerc cpuperc = sigar.getCpuPerc(); } catch (SigarException e) { e.printStackTrace(); } // percentage of CPU usage double cpuUsage = cpuperc.getCombined() * 100;