Revolutionizing Loan Approval: AI-Powered FastAPI and MindsDB Solution

Introduction

In the ever-evolving landscape of technology, Artificial Intelligence (AI) continues to reshape industries, offering innovative solutions to age-old problems. One such problem is the cumbersome and time-consuming process of loan approval. In this blog, we will explore the journey of creating a Loan Approval Agent using cutting-edge technologies such as FastAPI, React, PostgreSQL, and MindsDB, and how this solution addresses real-world challenges.

**What problem does LendSwift address?**

The traditional loan approval process is often plagued by inefficiencies, manual interventions, and a prolonged decision-making timeline. Banks and financial institutions grapple with the need to expedite this process while ensuring accuracy and compliance. The challenge was to develop a solution that streamline the loan approval process, reduces human errors, and enhances overall efficiency.

Inspiration✨

The inspiration behind this project stems from the recognition of the impact that AI can have on automating repetitive tasks, particularly in the financial sector. The goal was to leverage AI to create a system that could predict loan approval based on users data, offering a faster and more reliable alternative to the conventional approach.

Tech Stack

The chosen technology stack stack reflects a careful consideration of performance, scalability, and user experience:

FastAPI: A modern, fast web framework for building APIS with python.
React: A JavaScript library for building user interfaces, providing a responsive and dynamic frontend for a smooth user experience.
PostgreSQL: A robust relational database management system, ensuring secure storage and retrieval of user data.
MindsDB: MindsDB is an open-source platform that enables users to add Machine Learning capabilities to any database. MindsDB basically combines both AI and SQL functions in one; users can create, train, optimize, and deploy ML models without the need for external tools. Data analysts can create and visualize forecasts without having to navigate the complexities of ML pipelines.

Use Case

Imagine a user seeking a loan. They input their information. The Agent then processes this information using the MindsDB model trained on historical data to predict whether the loan should be approved or not. The user is promptly notified of the decision via email.

Key Features🚀

Predictive Analytics: MindsDB's machine learning capabilities enable the system to analyze historical data and make accurate predictions on loan approvals, improving decision making.
Real-time Communication: MindsDB's gmail handler allows for instant communication with users by sending automated emails containing the loan approval status, providing a seamless user experience.
Suggestions for Improvements: If the users predicted loan status is rejected, LendSwift will send some suggestions via email on how the user can enhance their chances of getting approved, based on their loan applications.
User Friendly Interface: The React based front end ensures an intuitive and aesthetically pleasing user interface, making it easy for users to input their data and understand loan approval decision.

Usefulness in Real World Scenarios

In the real world, the LendSwift offers several advantages:

Time Efficiency: The automated nature of the system accelerates the loan approval process, providing users with quick decisions, which is crucial in time-sensitive situations.
Reduced Human Error: By relying on AI for decision making the project minimizes the risk of human errors that may occur in manual loan approval processes.
Enhanced Customer Experience: The real time communication feature ensures that users are promptly informed of their loan status, improving overall customer satisfaction.
Data Driven Decision Making:MindsDB's machine learning model leverages historical data to make informed predictions, aiding financial institutions in making data driven decisions for loan approvals.

How exactly LendSwift works ?🤔

Now comes the most interesting part. What's the magic behind the LendSwift's predictions

A small note, I'll only paste the important codes here. If you want to check the full code, I'll leave the repo link at the end of this blog :)

Preparing Dataset: The magic behind the AI model is data. So the first thing that comes to mind when building a model is preparing and structuring the dataset like deleting or adding some columns, changing data types etc. After preparing the data its time to load the data into PostgreSQL.

import io
import requests
import pandas as pd
from env import config
from sqlalchemy import create_engine, text


#1LIvIdqdHDFEGnfzIgEh4L6GFirzsE3US
file_id = "1LIvIdqdHDFEGnfzIgEh4L6GFirzsE3US"
url = f"https://drive.google.com/uc?id={file_id}"

# Download the contents of the CSV file
download = requests.get(url).content

# Read the CSV file into a Pandas DataFrame
df = pd.read_csv(io.StringIO(download.decode("utf-8")))

DATABASE_URL= config("DATABASE_URL")

engine = create_engine(str(DATABASE_URL))

table_name = "loan_data"
out_df.to_sql(table_name, engine, if_exists = "replace", index = False)

Connecting PostgreSQL With MindsDB: The synergy between PostgreSQL and MindsDB is where the magic truly begins. Establishing a connection between the two allows MindsDB to tap into the rich dataset housed in PostgreSQL, setting the stage for intelligent model training.

After starting MindsDB, write the SQL query given below( off course with the actual data of your database ) in the editor to connect to your PostgreSQL database.

CREATE DATABASE psql_datasource
WITH
    ENGINE = 'postgres',
    PARAMETERS = {
    "host": "host",
    "port": 5432,
    "database": "postgres",
    "user": "postgres",
    "password": "password"
    };

Model Training Using MindsDB: The heart of LendSwift lies in the training of its AI model. MindsDB, with its robust REST API, becomes the conduit through which the model learns and evolves. The model ingests the structured dataset, gleaning patterns, and relationships that form the basis for making informed predictions. This training process is akin to imbuing the AI with the wisdom needed to discern potential loan approvals.

import requests

train_model = "loan_approval_predictor"
url = "http://127.0.0.1:47334/api/sql/query"

sql_command_2 = f"""
CREATE MODEL loan_agent.{train_model}
FROM test (
SELECT * FROM loan_data
)
PREDICT Loan_Status;
"""

commands_to_run = [ sql_command_2]

def mindsdb_query(url, sql_query):
    headers = {"Content-Type": "application/json"}
    return requests.post(url, json = {"query": sql_query}, headers = headers)

for cmd in commands_to_run:
    mindsdb_query(url, cmd)

After running this, head over to the MindsDB GUI, then click on the project's name. (By default, your project name is "mindsdb," but I've created my separate project named "loan_agent.") You can find a "models" folder with all the models listed under it.

You can also click on your model to see more details about it e.g. version, accuracy, status etc.

Making Predictions:
After the training is completed, LendSwift becomes highly proficient at making predictions. By utilizing MindsDB's REST API once again, the system can analyze the information users provide and provide precise answers regarding the approval status of their loan. Rest assured, if the loan status is not approved, our agent LendSwift will send some suggestions via email on how the user can enhance their chances of getting approved, based on their loan applications. This is the stage where the magic truly occurs, transforming information into practical and valuable outcomes.

import json
import uuid
import asyncio
import requests

MINDSDB_QUERY_ENDPOINT = "http://127.0.0.1:47334/api/sql/query"


async def mindsdb_query(MINDSDB_QUERY_ENDPOINT, sql_query):
    headers = {"Content-Type": "application/json"}
    response = await asyncio.to_thread(requests.post, MINDSDB_QUERY_ENDPOINT, json={"query": sql_query}, headers=headers)
    return response


async def predict_loan_approval(
    Email,
    Gender="Female",
    Married="No",
    Dependents="0.0",
    Education="Graduate",
    Self_Employed="No",
    ApplicantIncome="2900",
    CoapplicantIncome="0.0",
    LoanAmount="71.0",
    Loan_Amount_Term="360.0",
    Credit_History="1.0",
    Property_Area="Rural",
    raw_request=False
):

    message_id = str(uuid.uuid4())

    sql_query_2 = f"""
    CREATE JOB loan_agent.status_to_mail (
    CREATE OR REPLACE TABLE test.saved_prediction (
        SELECT m.Loan_Status, m.Email, m.id
        FROM loan_agent.loan_approval_predictor AS m
        JOIN test.loan_data AS d
    );
    );
    """

    response = await mindsdb_query(MINDSDB_QUERY_ENDPOINT, sql_query_2)

    await asyncio.sleep(10)

    sql_query_3 = f"""
    SELECT  Loan_Status, id
    FROM test.saved_prediction
    ORDER BY id DESC
    LIMIT 1;
    """

    response = await mindsdb_query(MINDSDB_QUERY_ENDPOINT, sql_query_3)
    response_text =  response.text 
    response_data = json.loads(response_text) 
    print(response_data) 
    loan_status = response_data['data'][0][0]
    print(loan_status)

    # Prepare the context string with user data
    context_data = f"Email: {Email}, Gender: {Gender}, Married: {Married}, Dependents: {Dependents}, " \
                   f"Education: {Education}, Self_Employed: {Self_Employed}, ApplicantIncome: {ApplicantIncome}, " \
                   f"CoapplicantIncome: {CoapplicantIncome}, LoanAmount: {LoanAmount}, " \
                   f"Loan_Amount_Term: {Loan_Amount_Term}, Credit_History: {Credit_History}, " \
                   f"Property_Area: {Property_Area}"
    question = "The lender has declined the users loan. Now give suggestions briefly to improve the users loan approval chance based on the given user data. Dont use the user word instead use your"

    sql_query_suggestions = f"""
    SELECT answer
    FROM loan_agent.lendswift
    WHERE question='{question}'
    AND context='{context_data}';
    """

    if loan_status == 'Y':
     subject = 'Loan Aprroved'
     body = 'Congratulations! Your loan data has potential to be approved.'
    else:
          # Execute the query and get the response
     response_suggestions = await mindsdb_query(MINDSDB_QUERY_ENDPOINT, sql_query_suggestions)
    #  suggestions = json.loads(response_suggestions.text)['data'][0][0]
     suggestions = json.loads(response_suggestions.text)['data'][0][0]
     body = suggestions.replace('"', '\\"')


     subject = 'Loan Rejected'





    sql_query_4 = f"""
INSERT INTO my_gmail.emails (message_id, to_email, subject, body)
VALUES ("{message_id}", "{Email}", "{subject}", "{body}");
"""

    response = await mindsdb_query(MINDSDB_QUERY_ENDPOINT, sql_query_4)

    print(response.text)
    return response

Then, this function is utilized by FastAPI to create the API. Finally, our React frontend uses this API to make predictions. Again if you want to check the complete code, I have left the repo link at the end of the blog.

In the above method we've only utilized Mindsdb's job statement to create our agent. Now we'll see how we can make our agent in a more simple way.

Working Agent with Chatbot

Now, let's delve into the world of amazing agents. Currently, agents can only be utilized by chatbots in Slack or Microsoft Teams, which is okay for now. Just imagine LendSwift on your WhatsApp direct messages. You won't have to go to banks or open a website (like the previous method). It will be right there on your WhatsApp. How cool would that be, right? However, WhatsApp integration with chatbots is a work in progress, so we'll be using Slack for now. You can learn more from here.

There's two ways to make an agent: Agents with Text-to-SQL skills, Agents with Knowledge Bases as skills.

Agents withText-to-SQL Skills: These are used when you want the agent to interact with and query a SQL database. You define a skill that enables the agent to convert natural language questions into SQL queries, which can then be executed against a specified database.
Agents withKnowledge Base Skills: These are suitable when you have a set of data or information that you want the agent to learn from and use for answering queries. This skill type involves creating a knowledge base and inserting data into it. The AI agent can then use this knowledge base to respond to queries. Here the data sources can be PDF and URL also.

For LendSwift we'll be going with Agents withText-to-SQL Skills as we've well structured dataset ready to go.

Creating a Model: Let's begin by creating a large conversational language model to be employed by an agent. Here we'll be using langchain as an engine but you can use openai if you want. The crucial aspect is to establish an effective prompt template.

def create_model():

    sql_query = f"""
CREATE MODEL my_model
PREDICT answer
USING
    engine = 'langchain_engine',
    question_column = 'question',
    api_key = '{API_KEY}', 
    user_column = 'question' ,
    assistant_column = 'answer',
    model_name='gpt-3.5-turbo', 
    verbose=true,
    mode = 'conversational',
    prompt_template='
Hi there, LendSwift here! <assume that you are a lender who can predict loan status using a simple decision making algorithm . user will give some data based on that you have to predict his loan status(dont need to think of some other factors). if your prediction says loan status will be approved then reply with " Congratulations! Your loan application has potential to be approved." or else reply with "We regret to inform you that your loan data lacks potential to be approved." providing cause for not approving and giving some suggestion what the user can improve. ';
"""
    res = mindsdb_query(MINDSDB_QUERY_ENDPOINT, sql_query)
    if res is not None:
        if res.status_code == 200:
            print("Model created successfully")
        else:
            print(
                f"Failed to create model. HTTP status code: {res.status_code}")
    else:
        print("Failed to create model. No response received.")

    create_model()

Creating skill: Now that our model is created lets create skill that will be used by the agent. We can create a skill using one or more tables from a connected data source.

def create_skill():

    sql_query = f"""
CREATE SKILL loan_skill
    USING
    type = 'text_to_sql',
    database = 'test', 
    tables = ['loan_data'];
    """
    res = mindsdb_query(MINDSDB_QUERY_ENDPOINT, sql_query)
    if res is not None:
        if res.status_code == 200:
            print("Skill created successfully")
        else:
            print(
                f"Failed to create skill. HTTP status code: {res.status_code}")
    else:
        print("Failed to create skill. No response received.")


    create_skill()

Creating the Agent: Now that we have both the model and skill let’s create an agent.

def create_agent():

    sql_query = f"""
CREATE AGENT my_loan_agent
USING
   model = 'my_model',
   skills = ['loan_skill'];
   """
    res = mindsdb_query(MINDSDB_QUERY_ENDPOINT, sql_query)
    if res is not None:
        if res.status_code == 200:
            print("Agent created successfully")
        else:
            print(
                f"Failed to create agent. HTTP status code: {res.status_code}")
    else:
        print("Failed to create agent. No response received.")

create_agent()

Creating the Chatbot: The next step would be to connect a chat app, like Slack, to MindsDB and create a chatbot utilizing this agent. You can follow this instruction to connect slack to Mindsdb.

def create_chatbot():

    sql_query = f"""
CREATE CHATBOT lendswift_chatbot
USING
    database = 'mindsdb_slack', 
    agent = 'my_loan_agent', 
    enable_dms = true;
    """

    res = mindsdb_query(MINDSDB_QUERY_ENDPOINT, sql_query)
    if res is not None:
        if res.status_code == 200:
            print("Chatbot created successfully")
        else:
            print(
                f"Failed to create chatbot. HTTP status code: {res.status_code}")

    else:
        print("Failed to create chatbot. No response received.")
create_chatbot()

If you've been following along so far, you should now have a functional agent integrated with a chatbot in Slack.

Repo link:click here

Demo video:click here

My Experience Throughout this Hackathon:

In concluding this exciting journey of creating LendSwift, it's essential to extend our heartfelt gratitude to Hashnode and MindsDB for organizing and hosting an exceptional hackathon. This platform has provided a space for innovation, collaboration, and learning, enabling developers to push the boundaries of what's possible in the realm of technology.

A special acknowledgment goes to MindsDB, this remarkable tool has democratized machine learning, making it accessible even to those with limited expertise in the field. MindsDB's user-friendly interface, coupled with its powerful capabilities, has empowered individuals like myself to delve into the world of machine learning and create impactful projects without an extensive background in the domain.

I teamed up with my friend, PROMIT BODAK to accomplish this project. We faced many hurdles but didn't stop. In the end, we made it work. Overall, it was a good learning experience.

Conclusion

The Loan Approval Agent( LendSwift ), powered by FastAPI, React, PostgreSQL, and MindsDB, represents a significant step towards revolutionizing the loan approval process. This innovative solution addresses the challenges posed by traditional methods, offering a faster, more accurate, and user-friendly alternative. As we move towards an era where AI becomes an integral part of everyday operations, projects like these showcase the transformative power of technology in solving real-world problems.

Biplab Bera's Blog