Employees can be deleted from the system only if their employee status is new. 

Navigate to Employee Management > Employees > Employee Details and the delete option will be available

Once you roll over into a new period and the status changes to Active and the Delete option will  be grayed out and wont be able to delete the employee.