Hi Lukasz,
You cannot increase the memory. The error you are getting is already a known error.
You can check the KBA : 2198778 - http500-javax.servlet.ServletException:#{setupWizardBean.initializePlugins}:javax.faces.el.EvaluationException received on system configuration wizard after login to CMC
Also I would recommend you to check with the DBA i.e. Database administrator to give DB_OWNER rights to the user from which you are connecting to the CMS database and allows account to use the CMS database with no restrictions.
Regards,
Mahak