Most input editor operations are now stored, and can be undone. This can be performed using contextual menu, shortcuts, or using the undo window which shows past actions and those which can be undone. Indeed, undo operations cannot modify inputs before the current frame.
The only operations that are not registered are when loading a savestate with its inputs. In that case, the undo history is reset.
This feature has been asked a lot. It is now possible to multiply the values of a single analog input (like mouse coordinates) by a factor. This will help porting inputs when the game window resolution has changed.
There is now a unified way to add input columns, especially analog inputs, using the “inputs” menu. You don’t to set a new value to an analog input to add it to the editor. As a consequence, using the mouse wheel to change its value has been removed.
It has a few new features:
A minimal hex viewer has been added, so that you can visualize the game memory. You can seek an address from the ram watch or ram search tools. Selecting bytes in the hex viewer will display the converted integer or float values. Also, modified bytes are highlighted.
As a critical part of the tool, savestating feature has received some improvements:
Files that are created or modified by the game (aka savefiles) were incorrectly left out of savestates. This is now fixed, but will cause an increase in savestate size. To mitigate this, a few tricks have been developed:
We map the savefile in memory. If the original file is present, we map it also. We compare the both memory regions, and we only store the memory pages that are different. When loading the savestate, we also map both files to memory, and we copy either from the original file or from the savestate into the savefile.
If the original file does not exist, we save the whole file into the savestate.
Mapped files can have holes in them. We can detect them using lseek() with
SEEK_DATA and SEEK_HOLE arguments, and skip pages which are inside a hole.
Calling lseek() requires that we have a file descriptor associated with the
mapped file. In the special case of shared files that were mapped and then
closed, we can still have access to a file descriptor inside /proc/self/map_files/.
Getting access to this directory requires CAP_CHECKPOINT_RESTORE since
Linux 5.9. Before that, it requires CAP_SYS_ADMIN. libTAS program now asks at
startup to add CAP_CHECKPOINT_RESTORE to the executable.
In the case that the game manipulates a file using only the file stream given
by fopen(), without ever needing the underlying file descriptor, we have a
strong optimization! Instead of opening the file, we map the file in memory
with private mapping, and we create a stream that gives access to the memory
mapping directly (using fopencookie()). The advantages are:
file-page flag from /proc/pid/pagemap file. This
way, we are sure to only save memory pages that have been modified from the
original file.This feature is crucial for programs like pcem, which modifies the image disk
of the virtual machine, possibly taking several gigabytes of memory.
When a savestate needs to be recreated, we are attempting to give it the same pid it has before. This is possible using two recent features:
clone3 syscall which has a set_tid optionHaving the same pid everytime can mitigate some game crashes.
]]>The libTAS game HUD was rewritten to use ImGui. It has some constraints to be used, so it will only appear for games using OpenGL, Vulkan or SDL2 renderer (> 2.0.18). For now, the previous elements were implemented (framecount, inputs, crosshair, lua, messages, ram watches). It is very easy to write for ImGui, so more will come.
A last feature is game window detach. When used, the game will not take the full window, but will appear inside an ImGui window.
A limitation from using savestates has existed for a very long time: the impossibility to load a savestate if threads have been created or destroyed. It was very annoying to users, and made several games impossible to TAS. Some features were implemented to mitigate this issue, mainly the backtrack savestate that is automatically saved after a thread was created or destroyed, to be able to go back as early as possible. Still it was not good enough.
So I went ahead and started working on savestate code being able to create or destroy threads. My main help was coming from DMTCP which has a similar feature, although this program is created the process and all the threads, so the destroy part was not handled.
This feature is still largely untested, but has shown good results on games that use threading a lot (like .NET games), with still crashes from time to time. Thanks to this feature, backtrack savestates and savestate invalidation are not needed anymore, so they were removed.
Terminating a thread must be performed when threads are being suspended, before
the actual state loading. Two methods are used to terminate a thread: the first
one is by calling pthread_cancel(), which is a relatively safe way to stop a
thread, but it relies on the thread hitting a cancellation point. If this
method times out, we signal the thread, and the signal handler calls
pthread_exit() (highly unsafe).
Creating a thread must be performed after the loading state has been performed
because the memory layout and content must be present when the thread is
created (especially the thread stack). We use the low-level clone() function
so that it does not mess with pthread library, and the thread function then
calls setcontext() to resume execution to where it was saved.
Input editor markers have been extended a bit with its own panel to handle markers. We can now edit a marker’s text and seek to its frame
Add several new lua functions:
memory.baseAddress() to get the base address of a file loaded in the game
process memoryruntime.saveState() and runtime.loadState() to save and load savestatesruntime.isFastForward() and runtime.setFastForward() to control
fast-forwardmemory.readcstring() to read a C-style string from memory into a lua stringgui.quad() to draw quadsgui.text() was extended to include horizontal/vertical alignment, font size
and optional monospace fontgui.window() to draw an ImGui window that may be movable.Additionally, a few modifications were made on the callbacks:
onPaint() callback is not called when the screen is not rendering (due to
either non-draw frame or skipping with fast-forward).onFrame() callback so it is executed before onPaint(), which
feels more natural.onInput() can now modify inputs even in playback mode, and the modifications
are stored inside the movie.-force-gfx-direct to the commandline options.--audio-driver ALSA to the commandline options.UnityPlayer_s.debug) is
present in the game. This is to locate the function that threads use to wait
for new jobs to execute: UnityClassic::Baselib_SystemFutex_Wait(). In the
new engines, locating loading threads is even easier because the engine does
assign a name to each thread. So we just have to look for Loading.Preload,
Loading.AsyncRe, Background Job. and Job.Worker [N].Lua scripting has been rewritten with a dedicated window for showing running scripts. Individual scripts can be enabled/disabled, and each script has its own context, so that functions/globals do not collide. A new callback system is present to allow multiple functions to be added to a single callback. Old scripts are still supported as being translated into the new system. Moreover, when users modify a running script, it is automatically refreshed.
libTAS menus grow more and more, and it was getting difficult for users to navigate them. Most of the settings are now moved their own window with different tabs, while tools and common settings that you may change during the run are kept in the main menu.
]]>The RAM search feature has been completely rewritten. Previously it was storing all results in memory, and too many results would crash the program. The new implementation is based on what Cheat Engine is doing:
The ability to change past inputs in the input editor, with an automated state loading and fast-forward, has been useful for the process of TAS making, but it suffered from bugs and limitations when interacting during the state loading. Those limitations and bugs were based on the fact that changing inputs using the input editor was done by the UI thread, while the rest of the input logic (reading from movie file or user keyboard, etc.) is performed in the main thread.
Now, all input modifications in the input editor are sent to the main thread. When they are processed by the main thread, it checks if the modification is valid (e.g. can’t change a past input). In the meantime, input modifications are shown as light gray/dark gray in the input editor. This change allows user to still submit input modifications during the state loading, without the risk of doing invalid changes. Also, a rewind cannot be triggered when another one is happening.
Having to set multiple options before running a game has always been an issue in libTAS. A few modifications have been made to help users with that:
LD_LIBRARY_PATH variable
accordingly, so that users won’t have to manually fill the library path.gettime()
function too many times during a single frame, a message is printed on the
terminal indicating the user that the corresponding time-tracking option may
be needed.2020.2.2f1) don’t work with
the deterministic hack, because of how Unity calls thread sync routines. It
basically bypasses libc functions and uses kernel futexes directly.