Piperine is a poorly water-soluble drug and its bioavailability is a great barrier for this drug. The objective of current work was to formulate and investigate a self-microemulsifying drug delivery system (SMEDDS) of piperine as well as modification of liquid SMEDDS into the solid SMEDDS. Solubility study of piperine was performed in various excipients. Optimum composition of the microemulsion region was identified by using the pseudo-ternary phase diagram. Liquid SMEDDS formulation was optimized by Box–Behnken statistical design. The optimized liquid SMEDDS formulation showed particle size 22.12 ± 0.88 nm, PDI 0.185 ± 0.03 and transmittance 99.48 ± 0.32%. A methyl thiazolyl tetrazolium (MTT) assay was performed to determine in vitro anticancer efficacy of piperine using KB cell lines. The piperine loaded self-microemulsifying nanoparticles significantly inhibited the growth of carcinoma cells by targeting mitochondria and produced caspases-3 activity. Whereas, this mechanism showed significant role in inhibition of cancer signaling stage. The MTT assay confirmed 98.7% cytotoxicity in cancerous cells. Moreover, PXRD analysis showed the transformation of the crystalline structure of piperine to the amorphous (dissolved) state. In vitro release and ex vivo permeation of liquid and solid-SMEDDS has shown faster drug release as compared to the pure drug. The developed formulation was stable under stability conditions after 3 months of storage. Thus, the studies revealed that SMEDDS has a promising strategy for a poorly water-soluble drug, which benefits to enhance the solubility, dissolution also an anti-proliferative performance of piperine in the oral squamous carcinoma.