Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 8 additions & 15 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ SharpSync is a **pure .NET file synchronization library** with no native depende
### Platform-Specific Optimizations

- **Nextcloud**: Native chunking v2 API support (fully implemented)
- **OCIS**: TUS protocol preparation (NOT YET IMPLEMENTED - falls back to generic upload)
- **OCIS**: TUS 1.0.0 protocol for resumable uploads (fully implemented)
- **Generic WebDAV**: Fallback with progress reporting

### Design Patterns
Expand Down Expand Up @@ -343,7 +343,6 @@ var deleted = await engine.ClearOperationHistoryAsync(DateTime.UtcNow.AddDays(-3
| Gap | Impact | Status |
|-----|--------|--------|
| Single-threaded engine | One sync at a time per instance | By design - create separate instances if needed |
| OCIS TUS not implemented | Falls back to generic upload | Planned for v1.0 |

### ✅ Resolved API Gaps

Expand All @@ -367,13 +366,11 @@ var deleted = await engine.ClearOperationHistoryAsync(DateTime.UtcNow.AddDays(-3

These APIs are required for v1.0 release to support Nimbus desktop client:

**Protocol Support:**
1. OCIS TUS protocol implementation (`WebDavStorage.cs:547` currently falls back)

**Progress & History:**
2. Per-file progress events (currently only per-sync-operation)
1. Per-file progress events (currently only per-sync-operation)

**✅ Completed:**
- OCIS TUS 1.0.0 protocol - Resumable uploads for OCIS servers with chunked transfer and fallback
- `GetRecentOperationsAsync()` - Operation history for activity feed with time filtering
- `ClearOperationHistoryAsync()` - Cleanup old operation history entries
- `CompletedOperation` model - Rich operation details with timing, success/failure, rename tracking
Expand Down Expand Up @@ -402,17 +399,17 @@ These APIs are required for v1.0 release to support Nimbus desktop client:
| Component | Score | Notes |
|-----------|-------|-------|
| Core sync engine | 9/10 | Production-ready, well-tested |
| Nextcloud WebDAV | 8/10 | Missing OCIS TUS protocol |
| Nextcloud WebDAV | 9/10 | Full support including OCIS TUS protocol |
| OAuth2 abstraction | 9/10 | Clean interface, Nimbus implements |
| UI binding (events) | 9/10 | Excellent progress/conflict events |
| Conflict resolution | 9/10 | Rich analysis, extensible callbacks |
| Selective sync | 10/10 | Complete: folder/file/incremental sync, batch notifications, rename tracking |
| Pause/Resume | 10/10 | Fully implemented with graceful pause points |
| Desktop integration hooks | 10/10 | Virtual file callback, bandwidth throttling, pause/resume, pending operations |

**Current Overall: 9.3/10** - Strong foundation with comprehensive desktop client APIs
**Current Overall: 9.5/10** - Production-ready with comprehensive desktop client APIs

**Target for v1.0: 9.5/10** - OCIS TUS and per-file progress remaining
**Target for v1.0: 9.7/10** - Per-file progress remaining

## Version 1.0 Release Readiness

Expand Down Expand Up @@ -464,15 +461,11 @@ All critical items have been resolved.
- BenchmarkDotNet suite for sync operations
- Helps track performance regressions

5. **OCIS TUS Protocol**
- Currently falls back to generic upload at `WebDavStorage.cs:547`
- Required for efficient large file uploads to ownCloud Infinite Scale

6. **Per-file Progress Events**
5. **Per-file Progress Events**
- Currently only per-sync-operation progress
- Would improve UI granularity for large file transfers

7. **Advanced Filtering (Regex Support)**
6. **Advanced Filtering (Regex Support)**
- Current glob patterns are sufficient for most use cases

### 📊 Quality Metrics for v1.0
Expand Down
4 changes: 4 additions & 0 deletions src/SharpSync/SharpSync.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@
<None Include="../../README.md" Pack="true" PackagePath="\" />
</ItemGroup>

<ItemGroup>
<InternalsVisibleTo Include="SharpSync.Tests" />
</ItemGroup>

<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.2" />
<PackageReference Include="sqlite-net-pcl" Version="1.9.172" />
Expand Down
10 changes: 10 additions & 0 deletions src/SharpSync/Storage/ServerCapabilities.cs
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,16 @@ public class ServerCapabilities {
/// </summary>
public bool SupportsOcisChunking { get; set; }

/// <summary>
/// TUS protocol version supported (e.g., "1.0.0")
/// </summary>
public string? TusVersion { get; set; }

/// <summary>
/// Maximum upload size supported by TUS endpoint
/// </summary>
public long? TusMaxSize { get; set; }

/// <summary>
/// Whether the server is a generic WebDAV server
/// </summary>
Expand Down
193 changes: 188 additions & 5 deletions src/SharpSync/Storage/WebDavStorage.cs
Original file line number Diff line number Diff line change
Expand Up @@ -539,13 +539,196 @@ await ExecuteWithRetry(async () => {
}

/// <summary>
/// OCIS-specific chunked upload
/// OCIS-specific chunked upload using TUS protocol
/// </summary>
private async Task WriteFileOcisChunkedAsync(string fullPath, string relativePath, Stream content, CancellationToken cancellationToken) {
// OCIS uses TUS protocol for resumable uploads
// For now, fall back to generic upload
// A full implementation would use TUS client
await WriteFileGenericAsync(fullPath, relativePath, content, cancellationToken);
try {
await WriteFileOcisTusAsync(fullPath, relativePath, content, cancellationToken);
} catch (Exception ex) when (ex is not OperationCanceledException) {
// Fallback to generic upload if TUS fails
if (content.CanSeek) {
content.Position = 0;
}
await WriteFileGenericAsync(fullPath, relativePath, content, cancellationToken);
}
}

/// <summary>
/// TUS protocol version used for OCIS uploads
/// </summary>
private const string TusProtocolVersion = "1.0.0";

/// <summary>
/// Implements TUS 1.0.0 protocol for resumable uploads to OCIS
/// </summary>
private async Task WriteFileOcisTusAsync(string fullPath, string relativePath, Stream content, CancellationToken cancellationToken) {
var totalSize = content.Length;
content.Position = 0;

// Report initial progress
RaiseProgressChanged(relativePath, 0, totalSize, StorageOperation.Upload);

// Create TUS upload
var uploadUrl = await TusCreateUploadAsync(fullPath, totalSize, relativePath, cancellationToken);

// Upload chunks
var offset = 0L;
var buffer = new byte[_chunkSize];

while (offset < totalSize) {
cancellationToken.ThrowIfCancellationRequested();

var remainingBytes = totalSize - offset;
var chunkSize = (int)Math.Min(_chunkSize, remainingBytes);

// Read chunk from content stream
content.Position = offset;
var bytesRead = await content.ReadAsync(buffer.AsMemory(0, chunkSize), cancellationToken);
if (bytesRead == 0) {
break;
}

try {
offset = await TusPatchChunkAsync(uploadUrl, buffer, bytesRead, offset, cancellationToken);
} catch (Exception ex) when (ex is not OperationCanceledException && IsRetriableException(ex)) {
// Try to resume by checking current offset
var currentOffset = await TusGetOffsetAsync(uploadUrl, cancellationToken);
if (currentOffset >= 0 && currentOffset <= totalSize) {
offset = currentOffset;
continue;
}
throw;
}

// Report progress
RaiseProgressChanged(relativePath, offset, totalSize, StorageOperation.Upload);
}
}

/// <summary>
/// Creates a TUS upload resource via POST request
/// </summary>
/// <returns>The upload URL from the Location header</returns>
private async Task<string> TusCreateUploadAsync(string fullPath, long totalSize, string relativePath, CancellationToken cancellationToken) {
using var httpClient = CreateTusHttpClient();

var request = new HttpRequestMessage(HttpMethod.Post, fullPath);
request.Headers.Add("Tus-Resumable", TusProtocolVersion);
request.Headers.Add("Upload-Length", totalSize.ToString());

// Encode filename in Upload-Metadata header
var filename = Path.GetFileName(relativePath);
var encodedMetadata = EncodeTusMetadata(filename);
request.Headers.Add("Upload-Metadata", encodedMetadata);

// Empty content for POST
request.Content = new ByteArrayContent(Array.Empty<byte>());

var response = await httpClient.SendAsync(request, cancellationToken);

if (!response.IsSuccessStatusCode) {
throw new HttpRequestException($"TUS upload creation failed: {(int)response.StatusCode} {response.ReasonPhrase}");
}

// Get upload URL from Location header
var locationHeader = response.Headers.Location;
if (locationHeader is null) {
throw new HttpRequestException("TUS server did not return Location header");
}

// Location may be absolute or relative
return locationHeader.IsAbsoluteUri
? locationHeader.ToString()
: new Uri(new Uri(_baseUrl), locationHeader).ToString();
}

/// <summary>
/// Uploads a chunk via TUS PATCH request
/// </summary>
/// <returns>The new offset after the chunk was uploaded</returns>
private async Task<long> TusPatchChunkAsync(string uploadUrl, byte[] buffer, int bytesRead, long currentOffset, CancellationToken cancellationToken) {
using var httpClient = CreateTusHttpClient();

var request = new HttpRequestMessage(HttpMethod.Patch, uploadUrl);
request.Headers.Add("Tus-Resumable", TusProtocolVersion);
request.Headers.Add("Upload-Offset", currentOffset.ToString());

// TUS requires application/offset+octet-stream content type
var content = new ByteArrayContent(buffer, 0, bytesRead);
content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/offset+octet-stream");
request.Content = content;

var response = await httpClient.SendAsync(request, cancellationToken);

if (!response.IsSuccessStatusCode) {
throw new HttpRequestException($"TUS chunk upload failed: {(int)response.StatusCode} {response.ReasonPhrase}");
}

// Get new offset from Upload-Offset header
if (response.Headers.TryGetValues("Upload-Offset", out var offsetValues)) {
var offsetStr = offsetValues.FirstOrDefault();
if (long.TryParse(offsetStr, out var newOffset)) {
return newOffset;
}
}

// If server doesn't return offset, calculate it ourselves
return currentOffset + bytesRead;
}

/// <summary>
/// Gets the current upload offset via TUS HEAD request (for resuming)
/// </summary>
/// <returns>The current offset, or -1 if the upload doesn't exist or is invalid</returns>
private async Task<long> TusGetOffsetAsync(string uploadUrl, CancellationToken cancellationToken) {
try {
using var httpClient = CreateTusHttpClient();

var request = new HttpRequestMessage(HttpMethod.Head, uploadUrl);
request.Headers.Add("Tus-Resumable", TusProtocolVersion);

var response = await httpClient.SendAsync(request, cancellationToken);

if (!response.IsSuccessStatusCode) {
return -1;
}

if (response.Headers.TryGetValues("Upload-Offset", out var offsetValues)) {
var offsetStr = offsetValues.FirstOrDefault();
if (long.TryParse(offsetStr, out var offset)) {
return offset;
}
}

return -1;
} catch {
return -1;
}
}

/// <summary>
/// Creates an HttpClient configured for TUS requests with OAuth2 authentication
/// </summary>
private HttpClient CreateTusHttpClient() {
var httpClient = new HttpClient {
Timeout = _timeout
};

// Add OAuth2 bearer token if available
if (_oauth2Result?.AccessToken is not null) {
httpClient.DefaultRequestHeaders.Authorization =
new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", _oauth2Result.AccessToken);
}

return httpClient;
}

/// <summary>
/// Encodes filename for TUS Upload-Metadata header (base64 encoded)
/// </summary>
internal static string EncodeTusMetadata(string filename) {
var encodedFilename = Convert.ToBase64String(Encoding.UTF8.GetBytes(filename));
return $"filename {encodedFilename}";
}

/// <summary>
Expand Down
Loading