Candle Breakout Analysis in MetaTrader 5 Using Python

Your Python script is designed to interact with the MetaTrader 5 (MT5) platform to perform the following tasks:

1. Connect to an MT5 Account: It initializes a connection to the MT5 terminal using specified login credentials.
2. Retrieve Historical Data: It fetches historical candle data for the "XAUUSD" asset within a specified date range.
3. Process and Analyze Data: It filters candles at specific times (20:29 and 20:30), calculates breakout statistics ("rompimentos"), and categorizes these breakouts into various ranges.
4. Save Results: It saves the retrieved candle data as a CSV file and the calculated statistics as a TXT file.

Below is a detailed breakdown of the script, along with recommendations for improvements and best practices.

1. Imports and Setup

import MetaTrader5 as mt5
import pandas as pd
from datetime import datetime, timedelta
import os
import sys

• MetaTrader5: Library to interact with the MT5 platform.
• Pandas: For data manipulation and analysis.
• Datetime: Handling date and time.
• OS & Sys: Interacting with the operating system and system-specific parameters.

2. Login Parameters

# Parâmetros de login
login = 1520378657  # Substitua pelo seu número de login
senha = "AA6=J?j67mU"  # Substitua pela sua senha
servidor = "FTMO-Demo2"  # Substitua pelo nome do servidor
mt5_path = r"C:\Program Files\FTMO MetaTrader 5\terminal64.exe"  # Caminho do MetaTrader 5

Security Concern:

• Hard-coded Credentials: Storing sensitive information like login credentials directly in the script is a security risk. If this script is shared or stored in a version-controlled repository, your credentials can be exposed.

Recommendation:

• Use Environment Variables: Store sensitive information in environment variables or use a configuration file with restricted access.

  import os

  login = os.getenv('MT5_LOGIN')
  senha = os.getenv('MT5_PASSWORD')
  servidor = os.getenv('MT5_SERVER')
  mt5_path = os.getenv('MT5_PATH', r"C:\Program Files\FTMO MetaTrader 5\terminal64.exe")

Then, set these environment variables in your system or use a .env file with libraries like python-dotenv.

3. Initialize and Login to MT5

# Inicializa a conexão com o MetaTrader 5
if not mt5.initialize(path=mt5_path):
    print("Falha ao inicializar o MetaTrader 5")
    mt5.shutdown()
    exit()

# Tenta fazer login na conta específica
if not mt5.login(login, password=senha, server=servidor):
    print("Falha ao fazer login no MetaTrader 5")
    print("Erro:", mt5.last_error())
    mt5.shutdown()
    exit()

# Verifica as informações da conta
conta_info = mt5.account_info()
if conta_info is None:
    print("Falha ao obter informações da conta")
    mt5.shutdown()
    exit()

print("Conectado à conta:", conta_info.login)
print("Nome do servidor:", conta_info.server)

Key Points:

• Initialization: Connects to the MT5 terminal using the specified path.
• Login: Attempts to log in with the provided credentials.
• Account Information: Retrieves and displays account details to confirm a successful connection.

Error Handling:

• The script gracefully handles failures in initialization, login, and account information retrieval by printing error messages and exiting.

4. Retrieve Historical Candle Data

# Define o ativo e o período de análise
ativo = "XAUUSD"  # Nome do ativo (Dow Jones)
periodo = mt5.TIMEFRAME_M1  # Timeframe de 1 minuto
data_inicio = datetime(2024, 7, 15)  # Data inicial
data_fim = datetime.now()  # Data final (hoje)

# Pega os dados históricos de candles entre as datas especificadas
dados_candles = mt5.copy_rates_range(ativo, periodo, data_inicio, data_fim)

# Converte os dados para um DataFrame para melhor visualização
df = pd.DataFrame(dados_candles)

# Verifica se o DataFrame está vazio
if df.empty:
    print("Nenhum dado foi retornado. Verifique a conexão com o MetaTrader 5 e a disponibilidade do ativo.")
    mt5.shutdown()
    exit()

# Exibe informações sobre o DataFrame
print("Formato do DataFrame:", df.shape)
print("Colunas do DataFrame:", df.columns)
print("Tipos de dados das colunas:")
print(df.dtypes)

# Salva o DataFrame como CSV para inspeção
df.to_csv('dados_XAUUSD_2029_2030.csv', index=False)
print("Dados salvos em 'dados_XAUUSD_2029_2030.csv'")

Functionality:

• Asset and Timeframe: Specifies the asset (XAUUSD) and the timeframe (1 minuto).
• Date Range: Fetches data from July 15, 2024, to the current date.
• Data Retrieval: Uses mt5.copy_rates_range to get historical candle data.
• DataFrame Conversion: Converts the retrieved data into a Pandas DataFrame for easier manipulation.
• Validation: Checks if the DataFrame is empty and exits if no data is returned.
• Saving Data: Saves the raw candle data as a CSV file for further inspection.

Recommendations:

• Dynamic Date Range: Instead of hardcoding data_inicio, consider making it dynamic or configurable.
• File Naming: The CSV file name dados_us30_1629_1630.csv might imply it contains data only for specific times. However, it currently contains data for the entire date range. Consider renaming it for clarity or adjusting the data being saved.

5. Processing Time Data

# Verifica se a coluna 'time' existe
if 'time' not in df.columns:
    print("A coluna 'time' não foi encontrada. Verifique o nome correto da coluna de tempo.")
    # Tenta encontrar uma coluna que possa ser a de tempo
    time_columns = df.select_dtypes(include=['int64', 'float64']).columns
    if len(time_columns) > 0:
        time_column = time_columns[0]
        print(f"Usando a coluna '{time_column}' como coluna de tempo.")
        df['time'] = pd.to_datetime(df[time_column], unit='s')
    else:
        print("Nenhuma coluna numérica encontrada para usar como tempo.")
        mt5.shutdown()
        exit()
else:
    df['time'] = pd.to_datetime(df['time'], unit='s')

Functionality:

• Time Column Verification: Ensures that the 'time' column exists in the DataFrame.
• Fallback Mechanism: If 'time' is missing, it attempts to find a numerical column to convert into datetime.
• Conversion: Converts the UNIX timestamp to a readable datetime format.

Recommendations:

• Assumption Check: The fallback assumes the first numerical column represents time, which might not always be accurate.
• Error Logging: Instead of printing messages, consider using logging libraries for better traceability.

6. Filtering Specific Candles

# Filtra os candles das 20:29 e 20:30 (horário MT5)
df_2029 = df[df['time'].dt.time == datetime.strptime("20:29", "%H:%M").time()]
df_2030 = df[df['time'].dt.time == datetime.strptime("20:30", "%H:%M").time()]

# Exibe os resultados filtrados
print("\nCandle das 20:29:")
print(df_2029)

print("\nCandle das 20:30:")
print(df_2030)

Functionality:

• Time Filtering: Filters the DataFrame to include only candles at 20:29 and 20:30.
• Display: Prints the filtered candles for inspection.

Recommendations:

• Time Zone Consideration: Ensure that the times correspond to the desired time zone. MT5 times are typically in the broker's server time, which might differ from your local time.
• Handling Missing Data: If no candles are found for these specific times on certain days, consider handling such cases to avoid index errors later.

7. Calculating Breakout Statistics

# Inicializa contadores para as estatísticas
stats = {
    '0': 0,
    '0-500': 0,
    '500-1000': 0,
    # ... (other ranges)
    '9500-10000': 0,
    '10000+': 0
}

# Inicializa contadores para as novas estatísticas
rompimentos_um_lado = 0
rompimentos_dois_lados = 0
sem_rompimentos = 0

# Inicializa uma lista para armazenar os detalhes dos rompimentos
detalhes_rompimentos = []

# Calcula os rompimentos do candle das 20:30 em relação ao das 20:29
for i in range(min(len(df_2029), len(df_2030))):
    data = df_2029.iloc[i]['time'].date()
    max_1629 = df_2029.iloc[i]['high']
    min_1629 = df_2029.iloc[i]['low']
    max_1630 = df_2030.iloc[i]['high']
    min_1630 = df_2030.iloc[i]['low']
    volume_1629 = df_2029.iloc[i]['tick_volume']
    volume_1630 = df_2030.iloc[i]['tick_volume']
    tamanho_2029 = (max_2029 - min_2029) * 100  # Multiplicado por 100
    tamanho_2030 = (max_2030 - min_2030) * 100  # Multiplicado por 100

    rompimento_baixo = (min_2029 - min_2030) * 100 if min_2030 < min_2029 else 0
    rompimento_cima = (max_2030 - max_2029) * 100 if max_2030 > max_2029 else 0

    # Calcula as diferenças
    diff_baixo = rompimento_baixo - tamanho_1630
    diff_cima = rompimento_cima - tamanho_1630

    # Adiciona os detalhes à lista
    detalhes_rompimentos.append({
        'DT': data,
        'RBaixo': rompimento_baixo,
        'DiffBaixo': diff_baixo,
        'RCima': rompimento_cima,
        'DiffCima': diff_cima,
        'V29': volume_2029,
        'V30': volume_2030,
        'T29': tamanho_2029,
        'T30': tamanho_2030
    })

    # Atualiza as estatísticas para rompimento baixo e cima
    for rompimento in [rompimento_baixo, rompimento_cima]:
        if rompimento == 0:
            stats['0'] += 1
        elif 0 < rompimento <= 500:
            stats['0-500'] += 1
        elif 500 < rompimento <= 1000:
            stats['500-1000'] += 1
        # ... (other ranges)
        elif 9500 < rompimento <= 10000:
            stats['9500-10000'] += 1
        else:
            stats['10000+'] += 1

    # Atualiza as novas estatísticas
    if rompimento_baixo > 0 and rompimento_cima > 0:
        rompimentos_dois_lados += 1
    elif rompimento_baixo > 0 or rompimento_cima > 0:
        rompimentos_um_lado += 1
    else:
        sem_rompimentos += 1

Functionality:

• Statistics Initialization: Sets up dictionaries and counters to categorize and count breakout events.
• Looping Through Candles: Iterates through each pair of 20:29 and 20:30 candles.
• Breakout Calculation:
  • Rompimento Baixo: Measures if the low of the 20:30 candle breaks below the 20:29 candle's low.
  • Rompimento Cima: Measures if the high of the 20:30 candle breaks above the 20:29 candle's high.
  • Differences: Calculates the difference between the breakout and the candle size.
• Statistics Update: Categorizes each breakout into predefined ranges and updates counters accordingly.
• Additional Statistics: Tracks whether breakouts occurred on one side, both sides, or none.

Recommendations:

• Vectorization for Efficiency: Instead of using a loop, consider vectorizing operations with Pandas for better performance, especially with large datasets.

  # Example of vectorized calculation
  df_combined = pd.merge(df_2029, df_2030, on='DT', suffixes=('_2029', '_2030'))
  df_combined['rompimento_baixo'] = (df_combined['low_2029'] - df_combined['low_2030']) * 100
  df_combined['rompimento_baixo'] = df_combined['rompimento_baixo'].apply(lambda x: x if x > 0 else 0)
  # Similarly for 'rompimento_cima' and other calculations

• Avoid Magic Numbers: The multiplication by 100 seems arbitrary. Document the reason or define it as a constant with a meaningful name.

  MULTIPLIER = 100
  tamanho_2029 = (max_2029 - min_2029) * MULTIPLIER
  # ...

• Range Definitions: The stats dictionary manually defines each range. Consider automating this or using tools like Pandas' cut function for scalability.

8. Generating and Saving Statistics

# Crie uma string com todas as estatísticas
estatisticas = f"""
Total de dias analisados: {len(df_2029)}

Detalhes dos Rompimentos:
"""

# Cabeçalho da tabela
estatisticas += f"{'DT':<12} {'RBaixo':>8} {'DiffB':>8} {'RCima':>8} {'DiffC':>8} {'V29':>6} {'V30':>6} {'T29':>8} {'T30':>8}\n"
estatisticas += "-" * 82 + "\n"

# Dados da tabela
for detalhe in detalhes_rompimentos:
    estatisticas += f"{detalhe['DT']!s:<12} {detalhe['RBaixo']:8.2f} {detalhe['DiffBaixo']:8.2f} {detalhe['RCima']:8.2f} {detalhe['DiffCima']:8.2f} {detalhe['V29']:6d} {detalhe['V30']:6d} {detalhe['T29']:8.2f} {detalhe['T30']:8.2f}\n"

estatisticas += f"""
Estatisticas de Rompimentos 20:29 a 20:30 :
Rompimentos = 0: {stats['0']}
Rompimentos > 0 e <= 500: {stats['0-500']}
# ... (other ranges)
Rompimentos > 10000: {stats['10000+']}

Estatisticas adicionais:
Candles rompidos somente de um lado: {rompimentos_um_lado}
Candles rompidos dos dois lados: {rompimentos_dois_lados}
Candles nao rompidos de nenhum lado: {sem_rompimentos}
"""

# Imprima as estatísticas no console
print(estatisticas)

Functionality:

• String Construction: Builds a formatted string containing all the calculated statistics.
• Table Formatting: Structures the details of each breakout in a tabular format.
• Summary Statistics: Provides a summary of breakouts across different ranges and additional categorization.

Recommendations:

• Use Pandas for Tables: Consider using Pandas' DataFrame capabilities to format and display tables, which can enhance readability.

  detalhes_df = pd.DataFrame(detalhes_rompimentos)
  print(detalhes_df.to_string(index=False))

• Localization: Ensure that the formatting (like decimal separators) aligns with your locale settings if sharing the output.

9. Saving Statistics to a Text File

# Obtenha o nome do arquivo atual e mude a extensão para .txt
nome_arquivo_atual = os.path.basename(sys.argv[0])
nome_arquivo_base = os.path.splitext(nome_arquivo_atual)[0]
nome_arquivo_txt = f"{nome_arquivo_base}.txt"

# Salve as estatísticas em um arquivo de texto
with open(nome_arquivo_txt, 'w') as arquivo:
    arquivo.write(estatisticas)

print(f"\nEstatisticas salvas no arquivo: {os.path.abspath(nome_arquivo_txt)}")

Functionality:

• File Naming: Derives the TXT file name from the current script's name.
• File Writing: Writes the constructed statistics string to the TXT file.
• Feedback: Prints the absolute path of the saved TXT file.

Recommendations:

• Output Directory: Specify an output directory to organize your output files, especially if running the script multiple times.

  output_dir = "output"
  os.makedirs(output_dir, exist_ok=True)
  nome_arquivo_txt = os.path.join(output_dir, f"{nome_arquivo_base}.txt")

• Error Handling: Handle potential file writing errors using try-except blocks.

  try:
      with open(nome_arquivo_txt, 'w') as arquivo:
          arquivo.write(estatisticas)
      print(f"\nEstatisticas salvas no arquivo: {os.path.abspath(nome_arquivo_txt)}")
  except IOError as e:
      print(f"Erro ao salvar o arquivo: {e}")

10. Shutdown MT5 Connection

# Desconecta o MetaTrader 5
mt5.shutdown()

Functionality:

• Cleanup: Properly closes the connection to the MT5 terminal to free resources.

Recommendation:

• Ensure Shutdown: Use finally blocks to guarantee that mt5.shutdown() is called even if errors occur earlier in the script.

  try:
      # ... (all your code)
  finally:
      mt5.shutdown()

11. General Recommendations and Best Practices

1. Error Handling:

   • Use Python's logging library instead of print statements for better control over log levels and outputs.
   • Implement comprehensive error handling to manage unexpected issues gracefully.

2. Modular Code:

   • Break down the script into functions or classes to enhance readability and maintainability.

    def initialize_mt5(mt5_path, login, senha, servidor):
        # Initialization and login code
        return conta_info
   
    def retrieve_candle_data(ativo, periodo, data_inicio, data_fim):
        # Data retrieval code
        return df
   
    def process_data(df):
        # Data processing and statistics calculation
        return estatisticas, detalhes_rompimentos
   
    # Main execution
    if __name__ == "__main__":
        # Call functions and manage workflow
   

3. Configuration Management:

   • Use configuration files (like YAML or JSON) or environment variables for settings and credentials.
   • This approach makes the script more flexible and secure.

4. Documentation:

   • Add docstrings and comments to explain the purpose of functions, parameters, and complex logic.
   • This practice aids future maintenance and collaboration.

5. Performance Optimization:

   • For large datasets, optimize data processing using vectorized operations with Pandas.
   • Profile the script to identify and address bottlenecks.

6. Testing:

   • Implement unit tests to ensure that each component of the script functions as expected.
   • Use testing frameworks like unittest or pytest.

7. Version Control:

   • Use Git or another version control system to track changes, collaborate, and manage different versions of your script.

8. Security:

   • Never commit sensitive information to version control.
   • Regularly update dependencies to patch known vulnerabilities.

12. Running the Script

To execute your script, follow these steps:

1. Ensure Prerequisites:

   • Python Installed: Ensure you have Python 3.x installed.
   • Install Required Libraries:

    pip install MetaTrader5 pandas
   

2. Configure Credentials:

   • Secure Credentials: As mentioned earlier, use environment variables or a secure configuration method to store your MT5 credentials.

3. Run the Script:
   

   python seu_script.py

Replace seu_script.py with the actual name of your Python script.

1. Check Output:
   • CSV File: Review dados_us30_2029_1630.csv for raw candle data.
   • TXT File: Review the generated .txt file for calculated statistics.

13. Potential Enhancements

1. User Input:

   • Allow users to input parameters like asset name, timeframe, date range, and specific times via command-line arguments or a configuration file.

2. Visualization:

   • Generate plots or charts to visualize breakout statistics using libraries like matplotlib or seaborn.

3. Automation:

   • Schedule the script to run at specific intervals using task schedulers like cron (Linux) or Task Scheduler (Windows).

4. Reporting:

   • Send the generated reports via email or integrate with dashboards for real-time monitoring.

5. Database Integration:

   • Store retrieved data and statistics in a database for persistent storage and more complex querying.

By implementing these recommendations, you can enhance the security, efficiency, and maintainability of your script, ensuring it performs reliably and scales with your needs.

If you encounter any specific issues or have further questions about particular parts of the script, feel free to ask!