Blockchain-Anchored Multi-Factor Deepfake Forensics for Trusted Identity Provenance

Blockchain-Anchored Multi-Factor Deepfake Forensics for Trusted Identity Provenance

1st Pradeep Kumar

Department of Computer Science Moradabad Institute of Technology

Moradabad, India

pradeep.mca11@gmail.com

2nd Utkarsh Saxena

Department of Computer Science Moradabad Institute of Technology

Moradabad, India

saxenautkarsh144@gmail.com

3rd Tanishka Ruhela

Department of Computer Science Moradabad Institute of Technology

Moradabad, India

tanishkaruhela512@gmail.com

4th Ujjwal Mishra

Department of Computer Science Moradabad Institute of Technology Moradabad, India

ujjawalmishra913@gmail.com

5th Manisha Kashyap

Department of Computer Science Moradabad Institute of Technology Moradabad, India

manishakashyap131400@gmail.com

AbstractThe rapid advancement of deepfake technology has made it extremely difcult for humans to differentiate between genuine and synthetically altered media. This limitation demands an automated, reliable, and secure detection method. To address this challenge, we propose a hybrid multi-factor framework that combines deep learningbased deepfake analysis with blockchain- supported identity and evidence verication. Our goal is not only to detect forged content in real time but also to ensure that the detection output and ownership metadata remain immutable, decentralized, and tamper-resistant.

For training and feature extraction, we employ the Xception network due to its efciency in learning micro-manipulation cues and its strong performance in image and video forensics. The model identies spatial and frequency-domain distortions, irregular facial transitions, boundary-level blending artifacts, and manipulation footprints. Along with spatial inconsistencies, temporal irregularities are captured through sequential frame correlation to detect motion discontinuity, expression instability, and biological pattern mismatches. Additionally, cross-modal synchronization validates whether speech rhythms align with lip movements, exposing phonemeviseme inconsistencies in forged videos.

To eliminate reliance on centralized authorities, detection records including condence score, hashed forensic signature, and timestamp are stored on an immutable decentralized ledger. This blockchain layer ensures evidence integrity, trans- parent traceability, and secure owner authentication.

The proposed framework demonstrates that integrating a lightweight deep learning backbone with blockchain-based ver- ication enhances trust, supports scalable deployment, and strengthens media authenticity auditing, contributing toward restoring digital media credibility.

Index TermsDeepfake detection, blockchain, deep learning, CNN, video authentication, smart contracts, multimedia security.

1. INTRODUCTION

   The rise of realistic synthetic media, commonly known as deepfakes, poses severe threats to privacy, security, and

   digital trust. Enabled by advanced generative models such as GANs and diffusion networks, deepfakes can convincingly manipulate facial expressions, voices, and entire video se- quences. These forgeries are increasingly being weaponized for misinformation campaigns, fraud, political manipulation, and reputational damage.

   Traditional deepfake detection methods primarily rely on deep learning models that analyse visual or audio artifacts in isolation. Although these methods achieve high accuracy on benchmark datasets, they often struggle to generalize to unseen manipulation techniques and can themselves be targeted by adversarial attacks. Furthermore, most existing pipelines treat detection as a stand-alone classication task, without consid- ering content provenance or long-term integrity of forensic evidence.

   To overcome these limitations, this work proposes a Blockchain and Deep Learning-Based Multi-Factor Frame- work for Real-Time Deepfake Detection. The core idea is to combine:

   • a deep learning-based detection engine to analyse media content,

   • a blockchain-based verication layer to ensure tamper- proof provenance,

   • and a multi-factor decision module that fuses AI predic- tions, on-chain validation, and metadata analysis into a holistic authenticity score.

   This hybrid architecture provides not only high-accuracy detection but also transparent, veriable, and tamper-resistant validation of digital media, making it suitable for deployment in high-risk environments such as journalism, law, defence, and nancial systems.

2. RELATED WORK

   Research on deepfake detection has evolved through multi- ple directions, each addressing specic aspects of media ma- nipulation. Deep convolutional neural networks such as Xcep- tionNet and EfcientNet have been widely adopted for their ability to capture ne-grained image features using depthwise separable and efcient convolutions. When trained on large- scale datasets like FaceForensics++, Celeb-DF, and DFDC, these models have demonstrated strong detection performance. With the rise of GANs and diffusion-based generators, spatio-temporal models and hybrid CNNRNN architectures were introduced to exploit temporal inconsistencies between frames. Multi-modal approaches combined audio and video, focusing on lip-sync, speechface alignment, and biometric consistency. While these methods improved robustness, they typically operated as centralized classiers and remained vulnerable to model tampering, dataset bias, and lack of

   transparent audit trails.

   Parallel to this, another line of work explored blockchain for digital content integrity. These systems store cryptographic hashes of original media, maintain audit trails, or certify capture events using secure hardware and smart contracts. However, most efforts focus primarily on provenance and not on real-time deepfake classication. Moreover, they rarely integrate AI-based analysis into the same trust pipeline.

   Overall, existing approaches either focus on deepfake de- tection or on blockchain-based content integrity, but seldom integrate both capabilities into a unied, multi-factor decision framework. This gap motivates the proposed architecture, which merges AI-based analysis, metadata inspection, and blockchain verication.

   TABLE I

   COMPARISON OF DEEPFAKE DETECTION APPROACHES

   Approach	Key Tech	Blockchain	Real- Time	Remarks
   Xception + FF++	CNN-based AI	No	Partial	High accuracy; no provenance check
   MesoNet	Shallow CNN	No	Yes	Fast inference; limited robustness
   EfcientNet	Efcient CNN	No	Partial	Accurate detection; metadata ignored
   Amber Video	Metadata + BC	Yes	No	Provenance- centric; lacks detection fusion
   Truepic	Secure Imag- ing + BC	Yes	No	Validates capture; no frame-level analysis
   Proposed Model	CNN + BC + M-F	Yes	Yes	Unied scor- ing; tamper- evident audit

3. PROPOSED FRAMEWORK

   To address the rising sophistication of deepfake generation methods, this study introduces a hybrid, real-time detection ar- chitecture that combines AI-driven analysis, distributed ledger verication, and metadata integrity assessment. Th integration of these components creates a robust pipeline capable of deliv- ering high detection precision while maintaining transparency and content authenticity.

   The framework consists of three major components:

   1. Deep learning-based detection engine,

   2. Blockchain-based verication layer,

   3. Multi-factor decision module.

   1. Deep Learning-Based Detection Engine

      This component acts as the primary detection layer, exam- ining visual content through state-of-the-art neural network architectures. The pipeline includes:

      • Frame Segmentation: Input videos are decomposed into individual frames using tools such as OpenCV, enabling detailed frame-level inspection.

        ×

      • Data Preprocessing: Each frame is standardized via resizing (e.g., to 224 224), normalization, and facial alignment to focus on key regions.

      • Model Prediction: A deepfake detection network such as Xception, EfcientNet, or Vision Transformers evaluates each frame and assigns a per-frame authenticity probability.

      • Temporal Fusion: Frame scores are aggregated using weighted averaging, voting, or sequence models to pro- duce a video-level decision.

      • Optional Multimodal Checks: For clips with audio, lip- sync verication, speaker identity matching, and cross- modal consistency checks can be applied.

        The detection engine is trained on heterogeneous datasets like Celeb-DF, DFDC, and FaceForensics++, allowing it to adapt to diverse manipulation styles and maintain robustness across varying quality levels.

        Fig. 1. Flowchart of Visual Defect Detection

   2. Blockchain-Based Verication Layer

      This layer introduces a transparent provenance mechanism that validates the originality of media using decentralized

      ledger technology. Blockchain serves as an immutable registry for storing and verifying authentic media signatures.

      • Video Hash Generation: Authentic videos are encoded into cryptographic hashes (e.g., SHA-256), producing unique, tamper-evident identiers.

      • Smart Contract Integration: Smart contracts written in Solidity are deployed on networks such as Ethereum or permissioned frameworks like Hyperledger Fabric. They handle registration and lookup of valid hashes.

      • On-Chain Verication: For a query video, its hash is computed and compared against on-chain records. A mis- match indicates unregistered or potentially manipulated content.

        Fig. 2. Conceptual working of the blockchain-based verication layer.

        Even if adversarial techniques manage to circumvent AI de- tectors, the blockchain ledger provides an additional safeguard by conrming whether media matches originally registered content.

   3. Multi-Factor Decision Module

   This nal module consolidates evidence from vari- ous sources to enhance reliability and reduce false posi- tives/negatives. It acts as the decision engine of the framework.

   1) Inputs: The module combines three data streams:

   1. AI Prediction Score (PAI): Deepfake probability from the CNN-based detection engine.

   2. Blockchain Validation (PBC): A binary indicator (1 = registered authentic hash found, 0 = not found).

   3. Metadata Analysis (PMeta): A score in [0, 1] derived from examining camera model, timestamps, geolocation, and compression history.

   1. Weighted Aggregation: A nal authenticity score Snal

      is computed as a weighted linear combination:

      Snal = wAIPAI + wBCPBC + wMetaPMeta, (1) where wAI + wBC + wMeta = 1. A typical conguration is

      wAI = 0.5, wBC = 0.3, and wMeta = 0.2.

   2. Decision Thresholds: Based on Snal:

   • Snal > 0.8 Classied as Authentic.

   TABLE II

   MODULE CONTRIBUTION OVERVIEW

   Module	Description	Output	Contribution
   CNN Detection	Spatial + artifact analysis	Prob. score	50%
   Blockchain Verify	Media hash vali- dation on chain	1/0	30%
   Metadata	Timestamp + camera + encoding	01	20%

4. IMPLEMENTATION

   Fig. 3. User interface of the Flask-based deepfake detection system, showing the video upload and analysis workow.

   Fig. 4. Confusion matrix and performance metrics (Precision, Recall, and F1-score) of the proposed deepfake detection model.

   The proposed multi-factor deepfake detection framework was implemented as a modular and scalable system, compris- ing three subsystems: the AI model, the blockchain backend, and the integration layer enabling real-time operation.

   1. Deep Learning Module: Model Training and Inference

      The deepfake detection engine was developed using Ten- sorFlow and Keras. A ne-tuned XceptionNet variant was trained on frames extracted from FaceForensics++ and DFDC datasets.

      The binary cross-entropy loss function was used:

      N

      N i i i i

      LBCE = 1 y log(y )+ (1 y ) log(1 y )], (2)

      { }

      i=1

      • Snal < 0.4 Classied as Deepfake.

      • 0.4 Snal 0.8 Manual Review required.

        where yi 0, 1 is the ground-truth label and yi is the predicted probability.

        Fig. 5. System-generated analysis report summarizing model prediction, metadata inspection, and blockchain verication outcome.

        Overall accuracy is dened as: Accuracy = TP + TN

        TP + TN + FP + FN

        , (3)

        Fig. 6. Dashboard of the proposed deepfake detection system displaying video input, prediction results, and verication details.

        where TP , TN , FP , and FN denote true positives, true negatives, false positives, and false negatives, respectively.

   2. Blockchain Module: Smart Contract Deployment

      The blockchain component was implemented in Solidity and deployed in a local Ganache environment for testing. Interac- tions with the contract were performed through Web3.py.

      Let H(V ) represent the SHA-256 hash of a video V :

      H(V ) = SHA256(V ). (4)

      A simplied smart contract function for hash verication is:

      function verifyHash(bytes32 hash) public view returns (bool) {

      return validHashes[hash];

      }

      The blockchain validation output PBC is:

      • PBC = 1, if H(V ) exists in the ledger,

      • PBC = 0, otherwise.

      Fig. 7. Detailed analysis report generated by the proposed system, presenting frame-level predictions, model condence scores, metadata summary, and blockchain verication results.

   3. Integration and Real-Time Testing

   A web-based proof-of-concept application was built using Flask as the backend. The complete ow is:

   Input Video Preprocessing CNN Inference SHA-256 Hashing Blockchain Check Metadata Extraction

   Final Decision

   Detection latency is modelled as:

   Ttotal = TAI + TBC + TMeta, (5)

   where TAI is AI inference time, TBC is blockchain lookup time, and TMeta is metadata extraction time.

   On a system with an NVIDIA RTX 3060 GPU and 16 GB RAM, for a 10-second video:

   • TAI 1.2 s,

   • TBC 0.8 s,

   • TMeta 0.5 s,

   iving Ttotal 2.5 s.

   TABLE III TECHNOLOGY STACK OVERVIEW

   Module	Technology Used	Description
   AI Model	TensorFlow, Keras	CNN-based deepfake detection
   Blockchain	Solidity, Ganache, Web3.py	Secure video hash registration & querying
   Web Integration	Flask, Python, OpenCV	End-to-end integration and frame handling
   Frontend (Optional)	HTML/CSS/JS	UI for video upload and visualiza- tion

5. RESULTS AND EVALUATION

   The proposed framework was evaluated on benchmark datasets and through real-time tests to assess accuracy, latency, and robustness.

   1. Model Performance

      The ne-tuned XceptionNet model achieved an average classication accuracy of 94.6% across a blended set of FaceForensics++ and DFDC videos.

      Key metrics:

      • Precision: 92.3%,

      • Recall: 95.1%,

      • F1-score: 93.7%,

      • AUC (Area Under ROC Curve): 0.96.

        These results demonstrate strong discrimination capability and balanced error rates, making the model suitable for practical deployment.

        Fig. 8. Architecture of the proposed multi-layered deepfake detection frame- work.

   2. Blockchain Overhead and Trust Enhancement

      The blockchain verication layer introduces a mean pro- cessing overhead of 150,ms180,ms, which remains highly efcient relative to deep learning inference latency in mul- timedia forensics. Despite its minimal temporal footprint, this layer fundamentally transforms the evidence pipeline by storing detection events as forensic transactions rather than centralized log records. To further strengthen evidence per- sistence and decentralized attestation, our framework anchors detection metadata using hashing mechanisms generated via the SHA-256 standard, ensuring cryptographic linkage be- tween media instances and verication signatures. For secure decentralized validation of forensic proofs, tamper detection, and ownership trail auditing, we utilize the scalable on-chain infrastructure supported by the Ethereum network, enabling automated integrity verication through deterministic smart contract execution. Tamper-evident signatures ensure that even minute alterations in deepfake media invalidate the forensic proof chain. This design balances performance and security to provide real-time trust assurance without affecting AI inference throughput. Experimental evaluations conrm that the added delay remains imperceptible to users while signif- icantly enhancing evidence accountability and decentralized verication reliability.

      \ { }

      In return, the system achieves notable improvements in: begin itemize [noitemsep]

      \

      item Tamper-evident data integrity using cryptographic signatures,

      \

      item Transparent auditability of detection and verication events,

      \

      item Increased user trust through veriable ownership provenance.

      \end{itemize}

   3. Multi-Factor Decision Boost

      The multi-factor decision module, combining AI scores, blockchain validation, and metadata, improved overall accu- racy by approximately 912% compared to standalone CNN- based detection.

      TABLE IV COMPARATIVE ACCURACY ANALYSIS

      Methodology	Accuracy (%)	F1-score	Latency (s)
      Standalone CNN Model (XceptionNet)	89.7	88.3	1.2
      CNN + Metadata	91.5	89.4	1.6
      Proposed Framework (Full System)	94.6	93.7	2.5

   4. Real-World Use Case Simulation

      In a Flask-based demo site for video uploads, the system:

      • Reliably detected tampered content, even under compres- sion,

      • Rejected unregistered content that could not be veried on-chain,

      • Provided explainable decisions via modular scores and logs.

   5. Robustness to Adversarial Content

   When evaluated against adversarially generated deepfakes designed to bypass conventional detectors, the proposed frame- work maintained detection accuracy above 90%, underscoring its robustness.

   Fig. 9. Performance comparison of detection models with and without blockchain and metadata integration.

6. DISCUSSION

   1. Multi-Layer Trust Mechanism

      Unlike conventional detectors that rely solely on a single AI model, the proposed framework implements defence-in-depth through three independent yet complementary layers:

      • AI-based visual analysis,

      • Blockchain-based provenance and non-repudiation,

      • Metadata-based contextual consistency checking.

        This synergy reduces the risk of any single point of failure and provides a more reliable and trustworthy decision pipeline.

   2. Scalability and Real-World Integration

      The modular design allows independent scaling of each layer and easy adaptation to different deployment scenarios:

      • Social media platforms: Server-side integration for pre- publication screening.

      • Content verication services: On-demand verication of user-submitted media.

      • Journalistic and legal domains: Forensic evidence val- idation and chain-of-custody preservation.

        The architecture can be deployed on cloud infrastructures or on-premise systems, depending on regulatory and privacy constraints.

   3. Limitations and Future Work

      Despite promising performance, several challenges remain:

      • AI Generalization: Accuracy may degrade for very low-quality videos or novel generative techniques not represented in training data.

      • Blockchain Scalability: High-throughput public net- works may require Layer-2 solutions or side-chains to scale.

      • Metadata Dependence: EXIF and encoding metadata can be stripped or forged, limiting reliability in some cases.

        Future work will focus on:

      • Self-supervised and adversarial training to improve gen- eralization,

      • Federated learning for privacy-preserving model updates,

      • Zero-knowledge proofs to enhance privacy of blockchain- based verication.

   4. Societal Implications

      In an era where deepfakes threaten elections, public trust, and personal safety, multi-factor verication systems such as the one proposed here can play a critical role in preserv- ing digital truth. By making detection and verication both technically sound and explainable, the framework supports regulators, platforms, and end-users in combating synthetic media abuse.

7. CONCLUSION

This paper introduced a novel, multi-layered framework for real-time deepfake detection and verication by strategically integrating articial intelligence, blockchain technology, and metadata analysis. The system addresses both sides of the challenge:

• Detection: Identifying sophisticated deepfakes via CNN- based models trained on diverse datasets.

• Verication: Validating originality and integrity using cryptographic hashes stored on a blockchain, comple- mented by metadata checks.

Experiments show that the integrated framework outper- forms standalone detection models, achieving 94.6% accuracy while keeping latency within practical limits for real-time ap- plications.The blockchain layer introduces minimal overhead but signicantly strengthens transparency and security.

Beyond the technical contributions, the framework lays foundations for broader content authentication ecosystems that can support social media moderation, news verication, legal evidence validation, and other high-stakes use cases. By combining multi-factor analysis with decentralized trust infrastructure, the proposed approach contributes to rebuilding condence in digital information and mitigating the harmful impact of synthetic media.

REFERENCES

1. R. Chesney and D. Citron, Deepfakes and the new disinformation war: The coming age of post-truth geopolitics, Foreign Affairs, vol. 98, no. 1, pp. 147155, 2019.

2. A. Rossler et al., FaceForensics++: Learning to detect manipulated facial images, in Proc. ICCV, 2019, pp. 111.

3. K. Dolhansky et al., The Deepfake Detection Challenge (DFDC) dataset, arXiv preprint arXiv:2006.07397, 2020.

4. N. Papernot et al., Practical black-box attacks against machine learn- ing, in Proc. AsiaCCS, 2017, pp. 506519.

5. S. Nakamoto, Bitcoin: A peer-to-peer electronic cash system, 2008. [Online]. Available: https://bitcoin.org

6. P. Kumar, R. Kumar, A. B. B. Abdul Hamid, and T. E. Nyamasvisva, Saket Application Methodology on Network Security with Blockchain Technology, in Recent Trends in Articial Intelligence and IoT, R. Kumar Tiwari and D. Singh, Eds., Communications in Computer and Information Science, vol. 2549. Springer, Cham, 2025.

7. Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu, Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics, in Proc. IEEE CVPR, 2020, pp. 32073216.

8. B. Dolhansky, J. Bitton, B. Paum et al., The Deepfake Detection Challenge Dataset, in Proc. IEEE CVPR, 2020, pp. 1048610495.

9. P. Kumar, R. Kumar, A. B. Abdul Hamid, and A. A. Elngar, A Novel Approach to Security Optimization in Distributed Cloud SaaS Using Serialized Sealing and Signing, LLM Nexus 2025 Online Conference on Large Language Models, New Delhi, India, vol. 10, Aug. 2025.

10. Z. Guo, G. Yang, J. Chen, and X. Sun, Deepfake video detection via multi-scale spatialtemporal networks, IEEE Trans. Multimedia, vol. 24, pp. 25062519, 2022.

11. X. Wang, Y. Wu, and P. Zhu, Face forgery detection by 3D convolu- tional neural networks, IEEE Trans. Information Forensics and Security, vol. 17, pp. 152165, 2022.

12. M. H. Nguyen, T. D. Nguyen, and S. R. Bhatia, A survey on deepfake detection techniques using deep learning, IEEE Access, vol. 11, pp. 3512035145, 2023.

13. A. Albahar and H. Alhussein, Deepfake detection using hybrid CNN and attention mechanisms, IEEE Access, vol. 11, pp. 7821578228, 2023.

14. S. Verdoliva, Media forensics and deepfakes: An overview, IEEE Journal of Selected Topics in Signal Processing, vol. 14, no. 5, pp. 910932, 2020.

15. R. Kumar, P. Kumar, and M. Kumar, Design Ubiquitous, Technolog- ically Efcient Online Storage System Using Blockchain, in Recent Trends in Articial Intelligence and IoT, R. Kumar Tiwari and D. Singh, Eds., Communications in Computer and Information Science, vol. 2549. Springer, Cham, 2025.

16. S. Hasan, A. Salah, and R. Jayaraman, Blockchain-enabled digital content authentication and provenance tracking, IEEE Access, vol. 11, pp. 4289142905, 2023.

17. R. K. Lomotey and R. Deters, Secure multimedia verication using blockchain technology, IEEE Trans. Multimedia, vol. 25, pp. 489501, 2023.

18. M. Chen, L. Zhao, and Y. Zhang, A blockchain-assisted framework for trustworthy multimedia forensics, IEEE Trans. Computational Social Systems, vol. 11, no. 1, pp. 3345, 2024.

19. J. Kim and S. Lee, Multi-factor authentication and trust modeling for digital media verication, IEEE Access, vol. 12, pp. 2154021555, 2024.

20. A. Sharma, V. Kumar, and P. Singh, Integrating deep learning and blockchain for secure deepfake detection, IEEE Access, vol. 13, pp. 1021110225, 2025.