Entering an Actual Start that is earlier than the Scheduled Start, say by 1 hour, is adding the 1 hour to the duration field instead of decrementing the finish date by 1 hour.
One office help entry states that "Entering an actual start or finish date for a task changes the corresponding scheduled or planned date for that task."
Perhaps by setting the actual start, you've changed the scheduled/planned start, but the scheduled/planned end stays the same, resulting in a longer duration (I would expect this behavior if the task is manually scheduled). Since you've entered it manually, Project may just think you're trying to edit the start only, and calculates the duration to match.
An alternative might be to change the scheduled/planned start to the actual date, and instead compare to baseline to show how the schedule has improved.