This study reports on the development of a gender-specific classification system able to discern between two levels of velocity of a caress-like stimulus, through information gathered from Autonomic Nervous System (ANS) linear and nonlinear dynamics. Specifically, caress-like stimuli were administered to 32 healthy volunteers (16 males) while monitoring electrocardiogram signal to extract Heart Rate Variability (HRV) series. Caressing stimuli were administered to the forearm at a fixed force level (6 N) and two levels of velocity, 9.4 mm/s and 37 mm/s. Standard HRV measures, defined in the time and frequency domain, as well as HRV nonlinear measures were extracted during the pre- and post-stimulus sessions, and given as an input to a Support Vector Machine (SVM) classifier implementing a leave-one-subject-out procedure. Results show an accuracy of velocity recognition of 70% for the men, and 84.38% for the women, when both standard and nonlinear HRV measures were taken into account. Conversely, non-significant results were achieved considering standard measures only, or a gender-aspecific classification. We can conclude that caress-like stimuli elicitation significantly affect HRV nonlinear dynamics with a highly specific gender dependency.