@aaronjheng Unfortunately, as described in the CLI documentation:
Most CLI parameters can be used headless, which means on a computer that is not configured to display a user interface. However, to export images or video an OS windowing system and OpenGL are required.
So it would be recommended to configure your AWS instances accordingly so that they support OpenGL rendering (you might need to change your configuration).
Otherwise, you could theoretically resort to software drivers like Mesa which emulate everything on the CPU (see this external forum thread). I doubt however that this will work at all and even if it does, it would be quite slow.