“small talk” is specific form of talk (talk that is of little consequence), not just talk that happens to be small.
“After a little more small talk, they went their separate ways.” should not be ‘corrected’ to “After a little more smaller talk, they went their separate ways.”