Initial commit
This commit is contained in:
gsinghpal
444
fusion_payroll/docs/PDF_FIELD_POSITIONING.md
Normal file
444
fusion_payroll/docs/PDF_FIELD_POSITIONING.md
Normal file
@@ -0,0 +1,444 @@
|
||||
# PDF Field Positioning System
|
||||
|
||||
## Overview
|
||||
|
||||
The PDF Field Positioning System is a dynamic configuration interface that allows users to define where text fields should be placed on flattened PDF templates (T4, T4 Summary, T4A, T4A Summary). This system was implemented to solve the problem of encrypted PDFs that couldn't be filled using traditional form-filling libraries.
|
||||
|
||||
Instead of using fillable PDF forms, the system overlays text directly onto flattened PDF templates using ReportLab, with user-configurable positions, font sizes, and font names.
|
||||
|
||||
**Status**: ✅ **IMPLEMENTED** (January 2025)
|
||||
|
||||
---
|
||||
|
||||
## Problem Statement
|
||||
|
||||
### Original Issue
|
||||
- Fillable PDF templates were encrypted, preventing automated form filling
|
||||
- Libraries like `pdfrw` and `PyPDF2` couldn't access form fields due to encryption
|
||||
- Error: "Permission error for the encrypted pdf"
|
||||
|
||||
### Solution
|
||||
- Use flattened (non-fillable) PDF templates
|
||||
- Overlay text directly onto PDF pages using ReportLab
|
||||
- Provide a user interface to configure field positions dynamically
|
||||
- Store positions in database for easy adjustment without code changes
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Model: `pdf.field.position`
|
||||
|
||||
**File**: `models/pdf_field_position.py`
|
||||
|
||||
Stores configuration for each field position on PDF templates.
|
||||
|
||||
#### Fields
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `template_type` | Selection | Yes | Template type: T4, T4 Summary, T4A, T4A Summary |
|
||||
| `field_name` | Char | Yes | Field identifier (e.g., "EmployeeLastName", "SIN", "Box14") |
|
||||
| `field_label` | Char | No | Human-readable label for display |
|
||||
| `x_position` | Float | Yes | X coordinate in points (default: 0.0) |
|
||||
| `y_position` | Float | Yes | Y coordinate in points (default: 0.0) |
|
||||
| `font_size` | Integer | Yes | Font size in points (default: 10) |
|
||||
| `font_name` | Char | Yes | Font family name (default: "Helvetica") |
|
||||
| `active` | Boolean | Yes | Whether this position is active (default: True) |
|
||||
| `sequence` | Integer | Yes | Display order (default: 10) |
|
||||
|
||||
#### Coordinate System
|
||||
|
||||
- **Origin**: Bottom-left corner of the page (0, 0)
|
||||
- **Units**: Points (1 point = 1/72 inch)
|
||||
- **Standard Letter Size**: 612 x 792 points (8.5" x 11")
|
||||
- **X-axis**: Increases to the right
|
||||
- **Y-axis**: Increases upward
|
||||
|
||||
#### Methods
|
||||
|
||||
##### `get_coordinates_dict(template_type)`
|
||||
|
||||
Returns a dictionary mapping field names to their position data.
|
||||
|
||||
**Parameters:**
|
||||
- `template_type` (str): Template type (e.g., "T4", "T4 Summary")
|
||||
|
||||
**Returns:**
|
||||
- `dict`: Format: `{'field_name': (x, y, font_size, font_name)}`
|
||||
- Only includes active positions
|
||||
|
||||
**Example:**
|
||||
```python
|
||||
coordinates = self.env['pdf.field.position'].get_coordinates_dict('T4')
|
||||
# Returns: {'EmployeeLastName': (100.0, 700.0, 12, 'Helvetica'), ...}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration with PDF Generation
|
||||
|
||||
### T4 Summary (`hr.t4.summary`)
|
||||
|
||||
**File**: `models/hr_payroll_t4.py`
|
||||
|
||||
#### Method: `_get_pdf_text_coordinates()`
|
||||
|
||||
Queries the `pdf.field.position` model for T4 Summary coordinates.
|
||||
|
||||
```python
|
||||
def _get_pdf_text_coordinates(self):
|
||||
"""Get text coordinates for PDF overlay from configuration."""
|
||||
return self.env['pdf.field.position'].get_coordinates_dict('T4 Summary')
|
||||
```
|
||||
|
||||
#### Method: `_overlay_text_on_pdf()`
|
||||
|
||||
Overlays text onto the PDF template using ReportLab.
|
||||
|
||||
**Process:**
|
||||
1. Loads flattened PDF template
|
||||
2. Retrieves coordinates from `_get_pdf_text_coordinates()`
|
||||
3. For each field in the data mapping:
|
||||
- Gets position data (x, y, font_size, font_name) from coordinates
|
||||
- Uses ReportLab's `canvas` to draw text at the specified position
|
||||
4. Returns base64-encoded filled PDF
|
||||
|
||||
**Key Code:**
|
||||
```python
|
||||
from reportlab.pdfgen import canvas
|
||||
from reportlab.lib.pagesizes import letter
|
||||
from PyPDF2 import PdfReader, PdfWriter
|
||||
import io
|
||||
|
||||
# Create canvas for text overlay
|
||||
packet = io.BytesIO()
|
||||
can = canvas.Canvas(packet, pagesize=letter)
|
||||
|
||||
# Get coordinates
|
||||
coordinates = self._get_pdf_text_coordinates()
|
||||
if not coordinates:
|
||||
raise UserError('No field positions configured. Please configure positions in: Payroll → Configuration → PDF Field Positions')
|
||||
|
||||
# Overlay text
|
||||
for field_name, value in field_mapping.items():
|
||||
if field_name in coordinates:
|
||||
x, y, font_size, font_name = coordinates[field_name]
|
||||
can.setFont(font_name, font_size)
|
||||
can.drawString(x, y, str(value))
|
||||
|
||||
can.save()
|
||||
```
|
||||
|
||||
### T4 Slip (`hr.t4.slip`)
|
||||
|
||||
**File**: `models/hr_payroll_t4.py`
|
||||
|
||||
Same pattern as T4 Summary, but queries for template type `'T4'`:
|
||||
|
||||
```python
|
||||
def _get_pdf_text_coordinates(self):
|
||||
return self.env['pdf.field.position'].get_coordinates_dict('T4')
|
||||
```
|
||||
|
||||
### T4A Summary (`hr.t4a.summary`)
|
||||
|
||||
**File**: `models/hr_payroll_t4a.py`
|
||||
|
||||
Queries for template type `'T4A Summary'`:
|
||||
|
||||
```python
|
||||
def _get_pdf_text_coordinates(self):
|
||||
return self.env['pdf.field.position'].get_coordinates_dict('T4A Summary')
|
||||
```
|
||||
|
||||
### T4A Slip (`hr.t4a.slip`)
|
||||
|
||||
**File**: `models/hr_payroll_t4a.py`
|
||||
|
||||
Queries for template type `'T4A'`:
|
||||
|
||||
```python
|
||||
def _get_pdf_text_coordinates(self):
|
||||
return self.env['pdf.field.position'].get_coordinates_dict('T4A')
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## User Interface
|
||||
|
||||
### Menu Location
|
||||
|
||||
**Path**: Payroll → Configuration → PDF Field Positions
|
||||
|
||||
**Menu Item ID**: `menu_fusion_pdf_field_positions`
|
||||
|
||||
**File**: `views/fusion_payroll_menus.xml`
|
||||
|
||||
### Views
|
||||
|
||||
#### List View (`pdf.field.position.list`)
|
||||
|
||||
**File**: `views/pdf_field_position_views.xml`
|
||||
|
||||
Displays all field positions in a table:
|
||||
- Template Type
|
||||
- Field Name
|
||||
- Field Label
|
||||
- X Position
|
||||
- Y Position
|
||||
- Font Size
|
||||
- Font Name
|
||||
- Active (toggle widget)
|
||||
- Sequence (handle widget for drag-and-drop ordering)
|
||||
|
||||
**Default Order**: `template_type, sequence, field_name`
|
||||
|
||||
#### Form View (`pdf.field.position.form`)
|
||||
|
||||
**File**: `views/pdf_field_position_views.xml`
|
||||
|
||||
**Sections:**
|
||||
1. **Header**: Active toggle
|
||||
2. **Basic Info**:
|
||||
- Template Type (required)
|
||||
- Field Name (required)
|
||||
- Field Label (optional)
|
||||
- Sequence (handle widget)
|
||||
3. **Position** (4-column layout):
|
||||
- X Position
|
||||
- Y Position
|
||||
- Font Size
|
||||
- Font Name
|
||||
4. **Info Alert**: Coordinate system explanation
|
||||
|
||||
#### Search View (`pdf.field.position.search`)
|
||||
|
||||
**File**: `views/pdf_field_position_views.xml`
|
||||
|
||||
**Searchable Fields:**
|
||||
- Template Type
|
||||
- Field Name
|
||||
|
||||
**Filters**: None (simplified for Odoo 19 compatibility)
|
||||
|
||||
---
|
||||
|
||||
## Security
|
||||
|
||||
### Access Rights
|
||||
|
||||
**File**: `security/ir.model.access.csv`
|
||||
|
||||
| ID | Name | Model | Group | Permissions |
|
||||
|----|------|-------|-------|-------------|
|
||||
| `pdf_field_position_hr_user` | PDF Field Position (HR User) | `pdf.field.position` | `hr.group_hr_user` | Read, Write, Create, Unlink |
|
||||
| `pdf_field_position_payroll_user` | PDF Field Position (Payroll User) | `pdf.field.position` | `hr_payroll.group_hr_payroll_user` | Read, Write, Create, Unlink |
|
||||
|
||||
---
|
||||
|
||||
## Module Manifest
|
||||
|
||||
**File**: `__manifest__.py`
|
||||
|
||||
The view file is included in the data list:
|
||||
|
||||
```python
|
||||
'data': [
|
||||
...
|
||||
'views/pdf_field_position_views.xml',
|
||||
...
|
||||
]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Usage Workflow
|
||||
|
||||
### 1. Configure Field Positions
|
||||
|
||||
1. Navigate to **Payroll → Configuration → PDF Field Positions**
|
||||
2. Click **Create**
|
||||
3. Fill in:
|
||||
- **Template Type**: Select T4, T4 Summary, T4A, or T4A Summary
|
||||
- **Field Name**: Enter the field identifier (must match the field name used in PDF generation code)
|
||||
- **Field Label**: Optional human-readable label
|
||||
- **X Position**: X coordinate in points
|
||||
- **Y Position**: Y coordinate in points
|
||||
- **Font Size**: Font size in points
|
||||
- **Font Name**: Font family (e.g., "Helvetica", "Times-Roman", "Courier")
|
||||
- **Active**: Check to enable this position
|
||||
- **Sequence**: Display order
|
||||
4. Click **Save**
|
||||
|
||||
### 2. Finding Coordinates
|
||||
|
||||
To determine X and Y coordinates:
|
||||
|
||||
1. Open the flattened PDF template in a PDF viewer
|
||||
2. Use a PDF coordinate tool or measure from bottom-left corner
|
||||
3. Convert inches to points: `points = inches × 72`
|
||||
4. For example:
|
||||
- 1 inch from left = 72 points
|
||||
- 1 inch from bottom = 72 points
|
||||
- Center of page (4.25" × 5.5") = (306, 396) points
|
||||
|
||||
### 3. Testing Positions
|
||||
|
||||
1. Configure a few test positions
|
||||
2. Generate a T4/T4A PDF
|
||||
3. Check if text appears in the correct location
|
||||
4. Adjust X/Y positions as needed
|
||||
5. Refine font size and font name for readability
|
||||
|
||||
### 4. Field Name Mapping
|
||||
|
||||
The `field_name` must match the keys used in the PDF generation code. Common field names:
|
||||
|
||||
**T4/T4 Summary:**
|
||||
- `EmployeeLastName`
|
||||
- `EmployeeFirstName`
|
||||
- `SIN`
|
||||
- `Box14` (Employment income)
|
||||
- `Box16` (CPP contributions)
|
||||
- `Box18` (EI premiums)
|
||||
- `Box22` (Income tax deducted)
|
||||
- `Box27` (CPP employer)
|
||||
- `Box19` (EI employer)
|
||||
- `PayerName`
|
||||
- `PayerAddress`
|
||||
- `TaxYear`
|
||||
|
||||
**T4A/T4A Summary:**
|
||||
- `RecipientLastName`
|
||||
- `RecipientFirstName`
|
||||
- `SIN`
|
||||
- `Box14` (Pension or superannuation)
|
||||
- `PayerName`
|
||||
- `PayerAddress`
|
||||
- `TaxYear`
|
||||
|
||||
---
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Dependencies
|
||||
|
||||
- **ReportLab**: For text overlay (`pip install reportlab`)
|
||||
- **PyPDF2**: For PDF manipulation (`pip install PyPDF2`)
|
||||
|
||||
### PDF Overlay Process
|
||||
|
||||
1. **Load Template**: Read flattened PDF using PyPDF2
|
||||
2. **Create Canvas**: Create ReportLab canvas for text overlay
|
||||
3. **Get Coordinates**: Query `pdf.field.position` model
|
||||
4. **Draw Text**: For each field, draw text at configured position
|
||||
5. **Merge**: Overlay text canvas onto PDF pages
|
||||
6. **Encode**: Convert to base64 for storage
|
||||
|
||||
### Error Handling
|
||||
|
||||
If no coordinates are found for a template type, the system raises a `UserError`:
|
||||
|
||||
```python
|
||||
raise UserError(
|
||||
'No field positions configured for this template type. '
|
||||
'Please configure positions in: Payroll → Configuration → PDF Field Positions'
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Issues Encountered & Resolved
|
||||
|
||||
### Issue 1: External ID Not Found
|
||||
|
||||
**Error**: `ValueError: External ID not found in the system: fusion_payroll.action_pdf_field_position`
|
||||
|
||||
**Cause**: Action definition order issue in XML files
|
||||
|
||||
**Resolution**: Ensured action is defined before menu item references it
|
||||
|
||||
### Issue 2: Invalid Model Name
|
||||
|
||||
**Error**: `ParseError: Invalid model name "pdf.field.position" in action definition`
|
||||
|
||||
**Cause**: Model not registered when XML was parsed
|
||||
|
||||
**Resolution**:
|
||||
- Verified model import in `models/__init__.py`
|
||||
- Ensured proper module loading order
|
||||
- Removed any syntax errors in model definition
|
||||
|
||||
### Issue 3: Invalid View Type
|
||||
|
||||
**Error**: `ParseError: Invalid view type: 'tree'`
|
||||
|
||||
**Cause**: Odoo 19 deprecated `tree` view type in favor of `list`
|
||||
|
||||
**Resolution**:
|
||||
- Changed `<tree>` to `<list>` in view definition
|
||||
- Updated view name from `pdf.field.position.tree` to `pdf.field.position.list`
|
||||
- Updated `view_mode` in action from `tree,form` to `list,form`
|
||||
|
||||
### Issue 4: Invalid Search View Definition
|
||||
|
||||
**Error**: `ParseError: Invalid view pdf.field.position.search definition`
|
||||
|
||||
**Cause**: Odoo 19 compatibility issue with search view structure
|
||||
|
||||
**Resolution**: Simplified search view to minimal structure (removed group filters)
|
||||
|
||||
---
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Potential Improvements
|
||||
|
||||
1. **Visual Position Editor**: Drag-and-drop interface to position fields visually
|
||||
2. **PDF Preview**: Preview PDF with current positions before saving
|
||||
3. **Bulk Import**: Import positions from CSV or JSON
|
||||
4. **Template Presets**: Pre-configured positions for common templates
|
||||
5. **Font Preview**: Preview font appearance before applying
|
||||
6. **Coordinate Calculator**: Tool to convert from inches/millimeters to points
|
||||
7. **Field Validation**: Validate field names against known field mappings
|
||||
8. **Version Control**: Track changes to positions over time
|
||||
|
||||
---
|
||||
|
||||
## Related Files
|
||||
|
||||
### Models
|
||||
- `models/pdf_field_position.py` - Model definition
|
||||
- `models/hr_payroll_t4.py` - T4/T4 Summary PDF generation
|
||||
- `models/hr_payroll_t4a.py` - T4A/T4A Summary PDF generation
|
||||
|
||||
### Views
|
||||
- `views/pdf_field_position_views.xml` - UI views
|
||||
- `views/fusion_payroll_menus.xml` - Menu integration
|
||||
|
||||
### Security
|
||||
- `security/ir.model.access.csv` - Access rights
|
||||
|
||||
### Manifest
|
||||
- `__manifest__.py` - Module configuration
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
|
||||
| Date | Changes |
|
||||
|------|---------|
|
||||
| 2025-01-XX | Initial implementation |
|
||||
| 2025-01-XX | Fixed Odoo 19 compatibility issues (tree→list, search view) |
|
||||
| 2025-01-XX | Removed debug instrumentation |
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- The system uses ReportLab's coordinate system (bottom-left origin)
|
||||
- Font names must match ReportLab's supported fonts (Helvetica, Times-Roman, Courier, etc.)
|
||||
- Positions are stored per template type, allowing different layouts for T4 vs T4A
|
||||
- Active flag allows temporarily disabling positions without deleting them
|
||||
- Sequence field enables drag-and-drop ordering in the list view
|
||||
Reference in New Issue
Block a user